Commit Graph

1292 Commits

Author SHA1 Message Date
Oswald Buddenhagen
fa8186c8d4 heed M_DEAD more thoroughly
this doesn't really fix anything under current conditions, as so far
only Maildir driver functions that modify known messages can cause
concurrent expunges to be detected, and we don't call any of these
before the so far unchecked loops.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
4e25fd59c1 fix possible attempts to set flags of M_DEAD messages
so far, we checked for M_DEAD only in loops over messages. but we should
have checked srec->msg uses as well. this would make the code a mess, so
instead call back from the drivers when messages are expunged, so we can
reset the pointers.

the only case where this really matters so far is the flag setting loop,
which may cause the concurrent expunge of not yet handled messages to be
detected by the maildir driver.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
87d1a4edde fix invalid data accesses in proxy driver callbacks
all printing of auxiliary callback arguments must be conditional on the
command having actually succeeded. this affected fetch_msg() most, which
outright crashed due to a null pointer deref.

to fix this for good, we automate the generation of the status printing
and checking. as a side effect, this fixes the fetch_msg() callback not
printing the status at all.

amends 4cc5ad5a.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
30a6015624 test also one-sided async operation
this tests only the common case of the far side being async - adding
100% instead of 50% to the runtime of the test to cover a corner case
didn't seem worth it.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
3a8f8a8391 fake async drivers more convincingly
instead of delaying the callback, delay the actual driver call. this is
in line with how the IMAP driver would behave, as since commit 6c08f568
it queues the socket writes (the network upstream latency goes on top,
but that doesn't alter the result).

amends 4423a932.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
16238909d3 print some more debug info during state reading and journal replay 2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
6e7b3d24c1 improve some debug messages
- print on which side we're upgrading a placeholder. as a side effect,
  this moves some magic out of upgrade_srec().
- don't use past tense for something that is only about to happen
- don't print status of every sync entry affected by journal replay -
  the entry load already prints it, and relevant operations print the
  new status. also, it was inconsistent with not printing the entry's
  old flags.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
950ebe833d add debug messages about effective requested operations 2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
3091e2fe5a add debug pretty-printing for OPEN_* flags as well 2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
17db5de0ca add debug pretty-printing for sync record status flags as well 2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
c902f69c6f format message flags in the sync debug output
now that we can do it cheaply, make copious use of it.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
a49017f481 streamline pretty-printing of message flags
wrap make_flags() into fmt_flags() which returns a (struct-wrapped)
string, so the calls can be inlined into the printf statements, without
reserving buffers.

we locally force optimization, so copy elision is always done, as debug
builds would otherwise suffer a somewhat unreasonable performance hit.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
a5dc1baedf denoise usages of INIT_IMAP_CMD()
move the variable declarations into the macro, taking advantage of the
fact that mixing them with code is ok since C99.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
f4ed8b27f6 fold superfluous INIT_IMAP_CMD_X() macro
it's been identical to INIT_IMAP_CMD() since c3d91ae1.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
f5d234ffa1 simplify an initialization in proxy driver code generator
the 'checked' hash entry doesn't exist yet at that point, so it's always
undefined.

amends 4423a932.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
5c44732fd9 support C99 comments in proxy driver code generator 2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
7f1c667910 cache drivers' ability to handle crlf in mails
it's a cheap call, but it clutters up log files.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
0f1b2b646b remove questionable optimization from case-insensitive string comparison
we optimized the case where the string would be equal even without
upper-casing, but for the much more common case where the strings differ
even after upper-casing, this was just an additional conditional.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
61b08880c8 improve tracking of maxxfuid
do it when the corresponding expiration events are logged/replayed.
that makes it unnecessary to log it separately just to make the
autotest happy.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
0f6362f2e2 fix tracking of sync record count through placeholder upgrades
for now, this is just pedantry, as it merely affects the size of the
already oversized { uid => srec } flathash after journal replay.

amends 70bad661.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
69118d25ec optimize { uid => srec } flathash
the sync records contain the uid, so storing it explicitly doubles the
hash's size for no good reason.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
8f4af5f78f make use of finduid[] after journal replay
we used to pass all initially loaded messages to match_tuids(), which
could be quite some when syncing old messages. as lost TUIDs result in
O(n^2) behavior, this could have a serious performance impact.
2022-06-19 16:03:44 +02:00
Oswald Buddenhagen
a1a3313ed4 print human-readable summary at exit
this is meant primarily for use with a redirected output, where no
progress counters are shown. but it would be weird not to have it with
tty output as well, so it replaces the counters after completion.

REFMAIL: 87bl2cgt6u.fsf@curie.anarc.at
2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
0f7c231cc2 fix bogus complaints about missing Store/Account references
... when the reference is simply invalid (we already get a complaint
about that).
2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
bf59636f0f don't stop validating Channel config after first error 2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
c986f80bb0 append even broken Channel configs to list
otherwise perfectly fine Group defs will complain about invalid refs.

we already do that with Store configs.
2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
2cbf8a68cf abort if any invalid Channels/Groups have been specified
this is more in line with expectations, and avoids a silly "No channel
specified." error message.
2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
44ad8f0361 handle mixing simple and compound sync options more explicitly 2022-06-19 16:03:38 +02:00
Oswald Buddenhagen
e70a20477c complain about --noop/--no-* conflicts
REFMAIL: 20211130124527.t3u7s4fyy57gmfzc@fastmail.com
2022-06-19 16:03:34 +02:00
Oswald Buddenhagen
be6e07c5c9 tone down complaints in merge_ops()
speak more accurately of redundancy, not conflict. these are fatal
errors at all only because the user likely meant something else than
they typed, so we force them to think again.
2022-06-19 16:03:34 +02:00
Oswald Buddenhagen
d7e3ae4b74 report location of overlapping operations 2022-06-19 16:03:34 +02:00
Oswald Buddenhagen
09f08e4974 stop parsing config lines after invalid keyword
otherwise we may pointlessly complain about excess tokens.
2022-06-19 16:03:34 +02:00
Oswald Buddenhagen
5d5e07eb63 add --list-stores mode
this is useful for verifying the store configuration, and finding the
right mailbox names.

REFMAIL: YaZC3XUTWjyfjgn+@ugly
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
d5a5da9475 factor out main_sync.c & main_p.h from main.c
this moves the channel iteration & synchronization code from main(),
with all its dependencies.

then it is "re-threaded" to be more directly driven by the driver
callbacks (like sync_boxes() is), rather than being a weird state
machine.

while the code is moved, localize many variables, and use an enum
instead of #defines for the states.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
01329bdf82 exclude placeholders from trashing
it makes no sense to trash the placeholders, and in fact the common
case is that they are deleted due to being replaced by the full
message.

a separate S_PURGED state needed to be added, as S_PURGE needs to be
reset after setting F_DELETED (so the operation doesn't count as still
pending after journal replay), yet trashing needs an indicator. logging
is now done via a separate command, as piggy-backing it on flag updates
just makes things less legible for no benefit.

this is mostly academical, as trashing being done on the side where
placeholders reside is rather unlikely.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
8363dbf2d1 add autotest for message trashing 2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
4b0c5a0cd5 do not exclude oversized messages from remote trashing
... as otherwise these messages would be just lost.

the assumption is that opposite-side trashing is used only for locally
generated messages whose size we control. it's also more consistent with
same-side trashing, where even oversized messages would be trashed.

the exclusion was broken anyway, as we failed to query the size of old
messages, particularly after 70bad661.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
d92c62022a treat not-really-propagated messages as trashable
messages for which only a placeholder was propagated should be treated
as not propagated, as otherwise the actual contents will be lost when
only not propagated messages are trashed.

amends 70bad661.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
f7650993b7 re-nest conditions for trashing messages
this makes the logic easier to follow and document in place.

also add some debug statements.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
91d19cceac make journal entry argument processing saner
the mega-if was unreadable and caused lots of churn.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
608c724add assert sizes of smaller-than-int bit fields in structures 2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
b3155a8bcb merge wstate back into status
this optimizes space usage, prospectively (we'd have to extend wstate
soon otherwise).

this partially reverts 4ffe1496.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
6a78e2c5f6 automate enumeration of power-of-two #defines
this is a lot more legible, and makes it possible to insert values in
the middle without churn.

i didn't find a way to do this with the pre-processor, so we now have
another code generator.

we now use the $< make variable, which requires gmake on netbsd < 9.0,
and possibly other systems with an ancient make.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
05122b678d print version and command line args in debug mode
that makes it unnecessary to ask help-seeking users for it separately.
2022-06-19 16:03:25 +02:00
Oswald Buddenhagen
c1eb3566b1 split Verbosity off from DFlags
this clearly documents the permitted states.
2022-06-19 16:03:21 +02:00
Oswald Buddenhagen
d3f118be79 re-interpret relative local paths in config file
this makes config+data file "sets" relocatable, which is useful for
testing.

this is technically a gratuitous backwards incompatible behavior
change, but to the degree that anyone uses relative paths at all, they
almost certainly rely on PWD being set up such that they won't see a
difference.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
cf13630a00 make default config & state locations follow the XDG basedir spec
there are fallbacks to the old locations for compatibility.

the new locations use "isync" instead of "mbsync", which is preparation
for renaming the executable back in v1.6.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
8bb679ea06 create sync state parent directories recursively
this may matter with SyncState being for example ~/.local/state/isync/.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
1ba0cd7b96 factor out sync_state.c & sync_p.h from sync.c
while moving the code, localize some variables, and use C99 comments.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
4b49848288 rearrange Makefile somewhat for consistency 2022-06-19 16:02:03 +02:00