Commit Graph

267 Commits

Author SHA1 Message Date
Oswald Buddenhagen
f2450cc4b8 centralize message flag pretty-printing somewhat
the flag values themselves are central, so we can centralize their most
common representation (Maildir's) just as well.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
4eff48c54e replace 1-t => t^1
this is consistent with what we already did for in-place operations.
arguably, it's also a bit more intuitive.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
08a375ea07 rename nfcalloc() => nfzalloc()
the signature is like that of zalloc() (as found in the linux kernel;
not to be confused with zone allocators, etc.), not like that of
calloc().
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
7d02d6c1fe move checked FILE functions to util.c
while they are used only in sync.c, they are conceptually low-level.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
6f023376a1 turn debug() functions into macros
this makes calling them with more expensive arguments efficient without
wrapping them into additional conditionals.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
1a0255c566 centralize some #includes
these are used (almost) everywhere, so put them in common.h.
2022-06-19 16:02:03 +02:00
Oswald Buddenhagen
98f4fd4586 shuffle around global variables
the new organization clarifies the ownership, making things more
self-contained. this will potentially help with unit testing.
2022-06-19 16:01:58 +02:00
Oswald Buddenhagen
0f2220634d coding style: add "superfluous" braces
- wrap flow-controlled statements that contain blocks into blocks
  themselves
- wrap bodies of do-while()s into blocks
- use braces on 'else' symmetrically (this obviously has a cascading
  effect, so this patch touches lots of lines)
- attach braces

unavoidably, the rules are sometimes broken around #ifdef-ery.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
156e9c5058 comment updates 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
254d2be9f4 remove pointless "lost track of ... messages" warnings
it doesn't really add any value when resuming after an interruption, as
incomplete propagations are perfectly expected, and are recovered from.
but it does make sense in the non-UIDPLUS path, where not finding just
stored messages indicates a problem.
so move the reporting from match_tuids() to msgs_found_new(), and remove
a redundant warning from the latter.

amends 6577bf3e.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
2b797fac61 delay TUID assignment less
we can't delay TUID assignment until after starting propagation if we
want to provide both safety and performance.

amends a0961d65.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
ae3a61b668 clarify / micro-optimize cancel_sync() 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
9356300952 convert licensing to SPDX
while at it, add/fix some licenses/copyrights/comments:
- it makes no sense to have a GPL exception in scripts
- ted did not contribute to the man page
- tst_timers is not part of the mbsync executable
- explicitly put the build system under GPL and add copyrights
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
72ba7ef125 bump copyrights
it's legally irrelevant, but whatever.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
7a4a887b3c sort lists of trashed messages after journal replay
the messages are trashed in mailbox (and thus UID) order, and in
practice we expect the operations to complete in order. however, if
older messages need to be trashed after a journal replay, and we get
interrupted again, the next replay would produce an unsorted array,
and thus break the binary search.

amends 2bba9b9.
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
c1feba585a don't clobber message status when upgrading placeholders
we'd reset the possibly set M_RECENT flag, which would lead to
pointless maildir rescans.

amends 70bad661.
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
e686f88318 don't complain about concurrent flagging as deleted
the result of propagating a deletion is flagging as deleted, so shut up
if the only remote change is exactly that.
2021-12-03 11:56:16 +01:00
Oswald Buddenhagen
51673214ab fix read beyond end of input in copy_msg_convert()
the input isn't necessarily null-terminated (it currently is for imap,
but not for maildir), so if the message ended somewhere within the
header field name, we'd read beyond its end, which theoretically could
cause a crash. no other adverse effects could result, as we'd stop
processing such a broken message right afterwards.

amends 70bad661.
2021-12-03 11:46:33 +01:00
Oswald Buddenhagen
92921b1d3b reject messages that grow too large due to conversion
that shouldn't really be a problem, as we have 2GB of headroom, and most
growth would happen when sending an all-newlines message from maildir to
imap (due to CR additions), which is mostly non-critical. but better
safe than sorry.
2021-12-01 10:07:40 +01:00
Oswald Buddenhagen
bc15e571b6 report conversion errors directly in copy_msg_convert()
that makes it easier to report various conditions without introducing
separate error codes.
2021-12-01 10:07:40 +01:00
Oswald Buddenhagen
87065c12b4 CVE-2021-44143: don't overflow heap on messages without headers
when a broken/compromised/malicious server gives us a message that
starts with an empty line, we'd enter the path for inserting a pristine
placeholder subject, for which we unfortunately didn't actually allocate
space (unless MaxSize is in use and the message exceeds it).

note that this cannot be triggered by merely receiving a crafted mail
with no headers (yes, it's actually possible to send such a thing), as
the delivery of mails adds plenty of headers.

amends 70bad661.
2021-11-25 16:14:32 +01:00
Oswald Buddenhagen
594e60bd74 make UIDVALIDITY recovery more strict about vanished messages
in particular, this covers the case of a mailbox being replaced with an
empty new one, which would subsequently lead to the opposite end being
emptied as well, which would typically be undesired.

also add plenty of comments.
2021-02-21 21:11:58 +01:00
Oswald Buddenhagen
6796e041ae improve error messages about irrecoverably changed UIDVALIDITY
don't print the actual values, which are meaningless technicalities
to the average user, and can be obtained separately for debugging if
really necessary.
also, fix the omission of the affected mailboxes from one of the
messages.
2021-02-21 21:11:58 +01:00
Oswald Buddenhagen
8457225a50 use more appropriate return value in driver_t::select_box()
don't say DRV_CANCELED when it's really DRV_STORE_BAD, as apart from
being just wrong, it lead to the confusing effect of canceling a store
as the result of a supposed cancellation of the same store.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
70bad66129 create placeholders for messages over MaxSize
this is vastly more useful than just omitting the messages with no
indication at all.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
68a412115a don't rewrite state gratuitously
delay the creation of the new state and journal until there is actually
something interesting to write. this saves some cpu cycles and prolongs
ssd life a whee bit.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
0e5046e14a add/fix/de-duplicate comments 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
fd7b5659ab de-duplicate updating of uid in sync records 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
e9efc49b6c do away with newmaxuid
now that expiration order is determined by a single loop ordered by
far-side UIDs, it is no longer necessary to accurately track the highest
seen UID.

as a side effect, this fixes a problem reported (way too long ago) by
Yuri D'Elia: we failed to up newmaxuid for messages we produced
ourselves, so we would keep enumerating the same messages until we also
propagated externally generated messages from that mailbox - which might
have been never for the server side of archive/trash mailboxes.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
de6dc699c9 make expiration loops solely far-side-driven
we can do that, as unpaired near-side messages are ignored anyway.

this mildly changes expiration order, as near-side messages that
existed for a long time but were propagated much later will be expired
later. however, that has no practical relevance.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
ca72383fe9 don't forget to skip dead messages on far side during expiration
this is mostly theoretical, as at this point no updates to the message
list can have actually happened. but it's future-proof and consistent
with the near-side loop.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
5d6741a9a8 streamline counting of currently pushed messages wrt. expiration
don't count them as alive just to ignore them in the next step.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
8df1f5dd64 re-nest conditions for syncing new messages
this makes the logic easier to follow and document in place.
also, make the comments actually match reality.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
ceb09fcd44 handle messages which are newly doomed after an interruption
we already didn't propagate messages which would be instantly expunged
from the target, but failed to cancel propagations that were already
scheduled before we got interrupted. this matters a bit when the
resumption happens significantly later than the initial attempt, giving
the user time to mark messages on the source as deleted.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
4aaada18e9 remove redundant condition
the 'pending' and 'skipped' sync record states are mutually exclusive
with having a complementary message, so there is no point in testing it
explicitly.

amends bd5fb6ff.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
395f802500 fix loading of some messages' sizes in some partial sync scenarios
we need to pass a different "boundary" UID to driver_t::load_box() for
every OPEN_* flag that queries a partial range:
- OPEN_FIND refers to messages newer than all we know about
- OPEN_OLD_IDS refers to messages which are paired
- OPEN_{OLD,NEW}_SIZE refers to messages (not) above the committed
  boundary of already propagated messages

we treated the 3rd like the 2nd, which was just wrong - the actual
boundary may be lower or higher, so we'd produce wrong results when
MaxSize was set and only one of New and ReNew was requested.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
c8f402e43f deprecate master/slave terminology
the underlying metaphor refers to an inhumane practice, so using it
casually is rightfully offensive to many people. it isn't even a
particularly apt metaphor, as it suggests a strict hierarchy that is
counter to mbsync's highly symmetrical mode of operation.

the far/near terminology has been chosen as the replacement, as it is a
natural fit for the push/pull terminology. on the downside, due to these
not being nouns, a few uses are a bit awkward, and several others had to
be amended to include 'side'. also, it's conceptually quite close to
remote/local, which matches the typical use case, but is maybe a bit too
suggestive of actually non-existing limitations.

the new f/n suffixes of the -C/-R/-X options clash with pre-existing
options, so direct concatenation of short options is even less practical
than before (some suffixes of -D already clashed), but doing that leads
to unreadable command lines anyway.

as with previous deprecations, all pre-existing command line and config
options keep working, but yield a warning. the state files are silently
upgraded.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
b514d9ddbc purge handling of pending sync entries from state file
these cannot actually end up in the committed state.

amends bd5fb6ff.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
d93726067d wrap jFprintf()+debug() into a macro
this ensures that everything that is logged to the journal also appears
in the debug output, and it makes the code less noisy.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
64e5f07ad3 consistently use NULL for null pointers
makes the code noisier, but also somewhat more expressive.
2020-08-04 17:16:01 +02:00
Oswald Buddenhagen
e2d3b4d55b fix lots of sign conversion warnings
... by making a lot of objects unsigned, and some signed.
casts which lose precision and change the sign in one go (ssize_t and
time_t to uint on LP64) are made explicit as well.
2020-08-04 17:15:39 +02:00
Oswald Buddenhagen
cc176df2c3 make some narrowing of integers explicit
this does specifically *not* cover about a bazillion warnings about
size_t being shrunk to uint - these make no sense given the expected
data set size.
2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
4d7e169e57 shrink some data at the source to avoid subsequent narrowing 2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
def22db096 constness fixes
add missing const qualifications, and add "const cast" suppressions
where unavoidable.
2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
5c2e8d3e14 make more objects static 2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
71d7d3e6df add some ATTR_* (mostly)
mostly ATTR_PRINTFLIKE(*, 0) for functions with a va_list argument.

also, one ATTR_NORETURN and one ATTR_UNUSED, both on functions.

also, an explicit suppression for a format string stored in a variable.
2020-08-04 17:13:56 +02:00
Oswald Buddenhagen
25b1c2b9e7 set sync record's flags only after propagating new message
this is semantically cleaner, and fixes storing the flags in the rare
case that flags are not being synced and the target is not being
expunged, as in this case flags are queried only during the actual
propagation.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
abdca388f6 atomize & document conditions in load() exception list construction 2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
b677bfe7e5 de-noise msg_copied() and flags_set() somewhat
assign temporary srec object instead of always spelling out the
indirection.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
841f07efd0 de-noise initialization of sync records
use calloc() instead of malloc().
2020-08-04 14:49:58 +02:00