Commit Graph

242 Commits

Author SHA1 Message Date
Oswald Buddenhagen
68a412115a don't rewrite state gratuitously
delay the creation of the new state and journal until there is actually
something interesting to write. this saves some cpu cycles and prolongs
ssd life a whee bit.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
0e5046e14a add/fix/de-duplicate comments 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
fd7b5659ab de-duplicate updating of uid in sync records 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
e9efc49b6c do away with newmaxuid
now that expiration order is determined by a single loop ordered by
far-side UIDs, it is no longer necessary to accurately track the highest
seen UID.

as a side effect, this fixes a problem reported (way too long ago) by
Yuri D'Elia: we failed to up newmaxuid for messages we produced
ourselves, so we would keep enumerating the same messages until we also
propagated externally generated messages from that mailbox - which might
have been never for the server side of archive/trash mailboxes.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
de6dc699c9 make expiration loops solely far-side-driven
we can do that, as unpaired near-side messages are ignored anyway.

this mildly changes expiration order, as near-side messages that
existed for a long time but were propagated much later will be expired
later. however, that has no practical relevance.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
ca72383fe9 don't forget to skip dead messages on far side during expiration
this is mostly theoretical, as at this point no updates to the message
list can have actually happened. but it's future-proof and consistent
with the near-side loop.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
5d6741a9a8 streamline counting of currently pushed messages wrt. expiration
don't count them as alive just to ignore them in the next step.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
8df1f5dd64 re-nest conditions for syncing new messages
this makes the logic easier to follow and document in place.
also, make the comments actually match reality.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
ceb09fcd44 handle messages which are newly doomed after an interruption
we already didn't propagate messages which would be instantly expunged
from the target, but failed to cancel propagations that were already
scheduled before we got interrupted. this matters a bit when the
resumption happens significantly later than the initial attempt, giving
the user time to mark messages on the source as deleted.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
4aaada18e9 remove redundant condition
the 'pending' and 'skipped' sync record states are mutually exclusive
with having a complementary message, so there is no point in testing it
explicitly.

amends bd5fb6ff.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
395f802500 fix loading of some messages' sizes in some partial sync scenarios
we need to pass a different "boundary" UID to driver_t::load_box() for
every OPEN_* flag that queries a partial range:
- OPEN_FIND refers to messages newer than all we know about
- OPEN_OLD_IDS refers to messages which are paired
- OPEN_{OLD,NEW}_SIZE refers to messages (not) above the committed
  boundary of already propagated messages

we treated the 3rd like the 2nd, which was just wrong - the actual
boundary may be lower or higher, so we'd produce wrong results when
MaxSize was set and only one of New and ReNew was requested.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
c8f402e43f deprecate master/slave terminology
the underlying metaphor refers to an inhumane practice, so using it
casually is rightfully offensive to many people. it isn't even a
particularly apt metaphor, as it suggests a strict hierarchy that is
counter to mbsync's highly symmetrical mode of operation.

the far/near terminology has been chosen as the replacement, as it is a
natural fit for the push/pull terminology. on the downside, due to these
not being nouns, a few uses are a bit awkward, and several others had to
be amended to include 'side'. also, it's conceptually quite close to
remote/local, which matches the typical use case, but is maybe a bit too
suggestive of actually non-existing limitations.

the new f/n suffixes of the -C/-R/-X options clash with pre-existing
options, so direct concatenation of short options is even less practical
than before (some suffixes of -D already clashed), but doing that leads
to unreadable command lines anyway.

as with previous deprecations, all pre-existing command line and config
options keep working, but yield a warning. the state files are silently
upgraded.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
b514d9ddbc purge handling of pending sync entries from state file
these cannot actually end up in the committed state.

amends bd5fb6ff.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
d93726067d wrap jFprintf()+debug() into a macro
this ensures that everything that is logged to the journal also appears
in the debug output, and it makes the code less noisy.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
64e5f07ad3 consistently use NULL for null pointers
makes the code noisier, but also somewhat more expressive.
2020-08-04 17:16:01 +02:00
Oswald Buddenhagen
e2d3b4d55b fix lots of sign conversion warnings
... by making a lot of objects unsigned, and some signed.
casts which lose precision and change the sign in one go (ssize_t and
time_t to uint on LP64) are made explicit as well.
2020-08-04 17:15:39 +02:00
Oswald Buddenhagen
cc176df2c3 make some narrowing of integers explicit
this does specifically *not* cover about a bazillion warnings about
size_t being shrunk to uint - these make no sense given the expected
data set size.
2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
4d7e169e57 shrink some data at the source to avoid subsequent narrowing 2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
def22db096 constness fixes
add missing const qualifications, and add "const cast" suppressions
where unavoidable.
2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
5c2e8d3e14 make more objects static 2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
71d7d3e6df add some ATTR_* (mostly)
mostly ATTR_PRINTFLIKE(*, 0) for functions with a va_list argument.

also, one ATTR_NORETURN and one ATTR_UNUSED, both on functions.

also, an explicit suppression for a format string stored in a variable.
2020-08-04 17:13:56 +02:00
Oswald Buddenhagen
25b1c2b9e7 set sync record's flags only after propagating new message
this is semantically cleaner, and fixes storing the flags in the rare
case that flags are not being synced and the target is not being
expunged, as in this case flags are queried only during the actual
propagation.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
abdca388f6 atomize & document conditions in load() exception list construction 2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
b677bfe7e5 de-noise msg_copied() and flags_set() somewhat
assign temporary srec object instead of always spelling out the
indirection.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
841f07efd0 de-noise initialization of sync records
use calloc() instead of malloc().
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
2f3cb5f481 fix signedness issues surrounding UIDs
amends bb632d1c.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
cab14608ca Merge branch '1.3' 2020-07-08 12:51:20 +02:00
Oswald Buddenhagen
96afe8d0c2 fix propagation of flagged oversized messages
... when not syncing flags and the target is not being expunged, as in
that case flags were not queried in time.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
e565d08246 don't try to propagate flags the target store does not support
$Forwarded is not standard, so it will most likely fail with mailboxes
that do not support keywords.

amends c4d7f018.
2020-01-08 18:22:48 +01:00
Michael J Gruber
c4d7f0189c implement Forwarded flag
maildir supports a 'P' flag which denotes the fact that a message has
been 'passed' on (forwarded, bounced). notmuch syncs this to the
'passed' tag.

Per https://tools.ietf.org/html/rfc5788, IMAP has a user-defined flag
(keyword) '$Forwarded' that is supported by many servers and clients
these days. (Technically, one should check for '$Forwarded' in the
server response.)

Restructure mbsync's flag parser to accept keywords (flags starting with
'$') but still bail out on unknown system flags (flags starting with '\').
Support '$Forwarded' as a first keyword since it maps to maildir's 'P'
and needs to be sorted in between the system flags.

Signed-off-by: Michael J Gruber <github@grubix.eu>
2018-07-01 12:36:28 +02:00
Michael J Gruber
e71f0ccc2a mark MAILBOX_DRIVER_FLAG locations in code
Mailbox driver flags are defined in several places. It is essential that
they are kept in sync, so mark them with the same string for easy
grepping with an alerting boiler plate.

Signed-off-by: Michael J Gruber <github@grubix.eu>
2018-07-01 12:30:59 +02:00
Oswald Buddenhagen
c29eceaeed make map_name() interpret empty strings as "no separator"
empty strings were previously meaningless, and starting with 72c2d695a,
failure to handle them lead to bogus results when the IMAP hierarchy
separator is legitimately empty (when the server genuinely supports none
and none is manually configured). non-null can be asserted more cleanly
than null-or-non-empty, so change the api like that.
incidentally, this also removes the need to work around gcc's bogus
warning in -Os mode.

problem found by "Casper Ti. Vector" <caspervector@gmail.com>
2017-10-15 16:53:27 +02:00
Oswald Buddenhagen
a5d4a0fe60 make sync records with stray TUID non-fatal
while the situation indicates an internal error, it is harmless in
itself. also, printing some more information may help identify the
problem.
2017-10-01 10:42:00 +02:00
Oswald Buddenhagen
bb632d1cd0 make UIDs unsigned
complies with the IMAP spec, thus removing the (not really) arbitrary
limitation to INT_MAX for UIDs.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
a0961d6505 delay assignment of TUID when propagating messages
go back to assigning TUIDs only right before actually propagating them.
this avoids spurious "TUID lost" warnings.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
bd5fb6fff3 move away from magic UIDs in the sync state
the only legitimate "deviant" UID is zero, meaning "no message". this
can be futher qualified by additional flags in the sync record, rather
than using magic values for the UID. in fact, the zero UID (so far
meaning only "expunged") was already optionally qualifed with "expired".

as a side effect, driver->store_msg() now returns 0 instead of -2 for
unknown UIDs. this was a hack to avoid translating the value later
on, but it made the api horrible, and now it's superflous in the first
place.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
4ffe149666 split off ephemeral sync record state to a separate member
this allows us to simplify logging of expiration, as we now can just log
the entire persistent state instead of fiddling with bits.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
efd72b85cc autotest: implement much more thorough resumption verification
the test will now make a test run for every journaled step, both right
before and right after the logging.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
4cc5ad5a1a introduce driver call debugging
do that by wrapping the actual stores into proxies.

the proxy driver's code is auto-generated from function templates, some
parameters, and the declarations of the driver functions themselves.
attempts to do it with CPP macros turned out to be a nightmare.
2017-04-22 11:26:11 +02:00
Oswald Buddenhagen
bbe4567bce let driver_t::openbox_box() return the UID validity
... and make 'uidvalidity' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
48ad58b9a3 use a #define for invalid UIDVALIDITY 2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
8d4918affd introduce get_uidnext() driver callback
... and make 'uidnext' private to the imap driver.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
683e581340 let driver_t::find_new_msgs() return the list of messages
consistently with driver_t::load_box().
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
74e9368121 let driver_t::load_box() return the list of messages
... and make 'msgs', 'count', and 'recent' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
c886f71054 make driver_t::prepare_load_box() return the final options
... and make 'opts' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
83ebe9022d introduce get_box_path() driver callback
... and make 'path' private to the maildir driver.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
d624c9af5d make set_bad_callback() a proper driver_t entry
... and make the pointers private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
f46cf8c887 provide a proper getter callback for driver capabilities
that way driver_t contains only callbacks.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
d54809e268 prepend "get_" to getters in driver_t
this makes it callbacks consistently start with a verb.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
36666f7e52 rewrite tracking of highest expired UID
so far, we tracked the slave side, and calculated the master side on the
fly. that complicated things unnecessarily.
2017-04-02 17:12:50 +02:00