Commit Graph

835 Commits

Author SHA1 Message Date
Oswald Buddenhagen
83adb9a39b actually implement maildir_get_uidnext()
the assumption was that this wouldn't be needed, as maildir_store_msg()
reliably delivers a UID. however, if we crash right before the callback
can record that UID, we'd still use OPEN_FIND in the next run, which
requires the saved next UID.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
c84235b068 get rid of maildir_find_new_msgs() stub
a failed assertion isn't any better than a clean segfault with an
obvious backtrace.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
38e7b4db22 Maildir: fix setting flags on messages without ":2," part
this is mostly hypothetical, as all messages i've encountered actually
have it even if no flags are set on them.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
406931bc45 autotest: use symbolic message subjects
... instead of numbers. otherwise there is too much confusion with UIDs.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
6734b9ce7d autotest: re-order mailbox contents according to UIDs
... instead of subjects, because that's way more natural and thus less
confusing.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
f3629c69e1 autotest: re-organize mailbox storage by UID
an effect of 7ce658d is that we can index messages by UID rather than
content (or more specifically, subject). apart from being cleaner, it
allows duplicated subjects.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
b59ee239a4 autotest: assume that each message has a UID
followup to 7ce658d14.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
50eab08509 autotest: show the right state file after idempotence test failure
amends efd72b85.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
d59392e901 autotest: fix prototype of ckbox() 2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
ef2caa074e autotest: create the temp dir in $TEMP
on modern systems, this makes it likely to end up on tmpfs, which is a
lot faster and ssd-friendlier.

the symlink is not deleted at the end, to minimize fs churn. that means
it will be dangling after a reboot, which gets fixed in the next run.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
5fee222f84 Merge branch '1.3' 2020-08-04 14:49:27 +02:00
Oswald Buddenhagen
30261fe6f1 fix version comparison in LibreSSL conditional
the operator was exactly inverted. that means that it actually wouldn't
compile with both older versions (that needed the aliases) and
potentially new versions (that will hide the data members - still not
the case as of 3.2).

amends 8a40554f0.
2020-08-04 14:42:42 +02:00
Oswald Buddenhagen
542e38dd49 fix re-using server connections for new stores
we failed to reset the box list pointer after freeing it, which would
lead to a crash.
we also failed to reset the listing status, which would lead to
malfunction if we hadn't already crashed.

this inlines imap_cleanup_store(), as there isn't much value in keeping
it. the message list is already freed when disowning the store anyway.
2020-08-04 14:42:42 +02:00
Oswald Buddenhagen
c82397cf6e don't crash in proxy_invoke_bad_callback()
we need to hold a ref to the proxy store, as after the bad_callback()
it's otherwise likely gone.
2020-08-04 14:40:19 +02:00
Oswald Buddenhagen
813ad67c56 fix simultaneously connecting to multiple hosts in non-IPv6 builds
we need to deep-copy the struct hostent data, as otherwise the
concurrent connects will overwrite each other's lookup results.

this is a rather hypothetical fix, as the bug currently affects only
channels connecting two IMAP accounts, and only if the first host's
first address asynchronously fails to connect.
2020-08-04 14:39:34 +02:00
Oswald Buddenhagen
3651c30296 increase PassCmd output buffer even more
apparently, some XOAUTH2 tokens are at 2.4K already, so make it 8K to be
*really* safe for a while.

REFMAIL: <20200716000515.GA2111668@lysator.liu.se>
2020-08-04 14:33:00 +02:00
Oswald Buddenhagen
09540b5648 unbreak CertificateFile documentation
the file may in fact contain CA certs.

amends 7d9d3e15.
2020-08-04 14:28:37 +02:00
Oswald Buddenhagen
cab14608ca Merge branch '1.3' 2020-07-08 12:51:20 +02:00
Oswald Buddenhagen
96afe8d0c2 fix propagation of flagged oversized messages
... when not syncing flags and the target is not being expunged, as in
that case flags were not queried in time.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
aff0c88a38 fix printf length arguments on lp64
found by coverity.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
04fc586e75 handle case-insensitivity of IMAP's INBOX
this is relevant only when listing an IMAP Store's contents, as that's
the only place where we aren't imposing the spelling ourselves.

we need to be careful not to treat our own canonical (prefix-stripped
and always slash-delimited) box names like that; codify that in
comments.

this reveals that commit 6f2160f1 may be deemed to have been incorrect -
the TODO item was ambiguous, and could quite possibly have meant this
fix. unsurprisingly, 380ccdd4 re-introduced it with more explicit
wording.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
94022a6752 catch server's failure to FETCH *
the query is untypical enough to have caused problems with davmail (when
we still used *:*) and mailo.com (until it got fixed), so better check
that the result (not) returned by the server makes sense.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
dfa8c16f27 don't timeout while uploading big messages
we did already set up the timeout when starting to send commands, but so
far we did not reset it when succeeding to send out data. rectify that.

REFFAIL: 87sgy92we3.fsf@jnanam.net
2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
121448ceb9 make -DC work with yama ptrace protection 2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
bee4fc54e7 fix overflows in uint comparisons 2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
a33e44758b fix roff abuse in man pages
".." is not valid. use "." instead, as recommended by groff_man(7).
this also necessitated adjusting the markup of the license blurbs.
2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
7d9d3e15f5 improve documentation of the server certificate related options 2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
48038fede1 mention --remove in --help output
amends d9a983a.
2020-07-08 11:12:59 +02:00
Oswald Buddenhagen
93fb3c29c6 fix UIDNEXT error message 2020-07-08 10:50:36 +02:00
Oswald Buddenhagen
e565d08246 don't try to propagate flags the target store does not support
$Forwarded is not standard, so it will most likely fail with mailboxes
that do not support keywords.

amends c4d7f018.
2020-01-08 18:22:48 +01:00
Oswald Buddenhagen
e8caaaaf65 rename F_PASSED => F_FORWARDED
the flag names are supposed to reflect the IMAP names (that's why
their #defines are followed by comments with the Maildir names).

amends c4d7f018.
2020-01-08 18:11:55 +01:00
Oswald Buddenhagen
462fed556a Merge branch '1.3' 2019-10-03 20:17:54 +02:00
Oswald Buddenhagen
d0a8551703 fix error handling of SSL_set_tlsext_host_name()
it is not one of the functions to be checked with SSL_get_error().

amends 17babc169.
2019-09-10 13:26:42 +02:00
Oswald Buddenhagen
6a874b5877 error-check more openssl function calls 2019-09-10 13:26:42 +02:00
Oswald Buddenhagen
5f908b7672 attempt to improve ssl error handling (again)
the error queue may contain multiple errors, so make sure to drain it
completely. also, don't try to fall back to errno in case of
SSL_ERROR_SSL.
2019-09-10 13:26:42 +02:00
Oswald Buddenhagen
91abf2b830 modernize ssl context init
with openssl 1.1+, use TLS_client_method() instead of the deprecated
SSLv23_client_method().
2019-09-10 13:26:42 +02:00
Oswald Buddenhagen
f63e4338e8 fix leak of openssl X509 objects
SSL_get_peer_certificate() increments the refcount of the object.
2019-09-10 13:26:42 +02:00
Oswald Buddenhagen
36062c5220 wording fixes in mbsync.1
inspired by Ulrich Ölmann <u.oelmann@pengutronix.de>.
2019-09-10 13:18:49 +02:00
Caspar Schutijser
7607e53d56 Do not crash when using Tunnel in an IPv6-enabled build
socket_connected() is also called on the tunnel pipe.

amends 3ceb55310.
2019-08-19 13:23:03 +02:00
Dmitry Torokhov
bf14798700 Bump up PassCmd buffer size to 2KiB
While ordinary passwords are rarely longer than 80 bytes, XOAUTH2 tokens
easily exceed this limit. Let's bump it up to 2K to be really safe.
2019-07-19 12:58:04 +02:00
Oswald Buddenhagen
fbc432aace fix parsing of NIL hierarchy delimiters in IMAP LIST responses
a server which does not support hierarchical mailboxes (e.g., seznam.cz
as of oct 2018) can legitimately send NIL (rather than an empty string).
2019-05-28 17:27:09 +02:00
Oswald Buddenhagen
702b6ec4a1 fix spacing in "SASL mechanism(s) not available" error message
amends fdb03b91.
2019-03-10 11:30:05 +01:00
Klemens Nanni
d61f462039 Fix CopyArrivalDate on platforms without glibc
strptime(3)'s "%d" day of the month conversion specifier does not accept
leading blanks in case of single digit numbers.  "%e" does that.

While implementation details and differences between the two
day-of-month conversion specifiers vary, none of the major libcs
(incl. OpenBSD, FreeBSD, Illumos, musl) consume a leading blank for "%d"
except glibc, which consumes any number of spaces like in the "%e" case.

Using "%e" ensures that date strings like " 4-Mar-2018 16:49:25 -0500"
are successfully parsed by all major implementations in compliance to
X/Open Portability Guide Issue 4, Version 2 ("XPG4.2").  musl is now the
only one that still treats "%d" and "%e" without stripping any space.

Issue analysed and reported by Evan Silberman <evan@jklol.net> who found
mbsync 1.3.0 on OpenBSD 6.4 to fail with `CopyArrivalDate' set when
syncing mails with the above mentioned timestamp.

See https://marc.info/?l=openbsd-tech&m=155044284526535 for details.
2019-02-20 23:02:02 +01:00
Gergely Risko
a8f6eebdd9 Work around useless SASL warnings
Ater sasl_client_step() is called and the Cyrus SASL library forwards
it to the client plugin, if the result value is OK (authentication
succeeded), the clientout is filled out to be an empty string, even if
the client plugin wanted to return NULL.

To avoid that mbsync complains at this point, check the returned length
instead of the pointer.
2019-02-05 11:29:13 +01:00
Oswald Buddenhagen
b72800944c fix formatting of uint in callback debug stubs
amends bb632d1c.
2019-01-05 00:00:10 +01:00
Oswald Buddenhagen
95d18e2778 Merge branch '1.3' 2018-11-27 00:51:03 +01:00
Oswald Buddenhagen
acfa3a2bbc sort messages from UID FETCH request
turns out that some IMAP servers (e.g., poczta.o2.pl) do not return
messages in ascending UID order in response to a UID FETCH request,
which makes the driver violate the API contract.

counter this by sorting the messages. this also addresses the
long-standing (but hypothetical) issue that parallel UID FETCH requests
could be handled out-of-order and thus also lead to mixed up results.

based on patch by Marcin Niestroj <macius1990w@gmail.com>.
2018-11-24 14:36:31 +01:00
Oswald Buddenhagen
f7cec3064d bump IMAP command buffer size to 4KiB
while only 1KiB is required by the IMAP spec, AUTHENTICATE GSSAPI with
Kerberos requires about 1700 bytes.
accomodate that, plus some reserve.

fix suggested by Tollef Fog Heen <tfheen@err.no> via Debian BTS.
2018-09-09 13:43:53 +02:00
Oswald Buddenhagen
17babc1695 use SNI when connecting with SSL
based on patch by Vincent Bernat <vincent@bernat.ch>.
2018-09-08 18:37:15 +02:00
Oswald Buddenhagen
37feeddbfb Merge branch '1.3' 2018-07-01 13:26:10 +02:00
Oswald Buddenhagen
f698f16967 fix type of 'port' and check its range in config reader 2018-07-01 13:25:16 +02:00
Klemens Nanni
470210fa86 Fix time_t format strings
For time_t, long long handles dates after Y2038 and should be safe on
32-bit architectures.

From Jeremie Courreges-Anglas <jca@openbsd.org>.
2018-07-01 13:24:59 +02:00
Klemens Nanni
8a40554f07 User functions provided by recent LibreSSL versions
At least on OpenBSD, this enables new APIs out of the box provided by
LibreSSL 2.7.1 and higher.

From Jeremie Courreges-Anglas <jca@openbsd.org>.
2018-07-01 12:44:19 +02:00
Michael J Gruber
c4d7f0189c implement Forwarded flag
maildir supports a 'P' flag which denotes the fact that a message has
been 'passed' on (forwarded, bounced). notmuch syncs this to the
'passed' tag.

Per https://tools.ietf.org/html/rfc5788, IMAP has a user-defined flag
(keyword) '$Forwarded' that is supported by many servers and clients
these days. (Technically, one should check for '$Forwarded' in the
server response.)

Restructure mbsync's flag parser to accept keywords (flags starting with
'$') but still bail out on unknown system flags (flags starting with '\').
Support '$Forwarded' as a first keyword since it maps to maildir's 'P'
and needs to be sorted in between the system flags.

Signed-off-by: Michael J Gruber <github@grubix.eu>
2018-07-01 12:36:28 +02:00
Michael J Gruber
e71f0ccc2a mark MAILBOX_DRIVER_FLAG locations in code
Mailbox driver flags are defined in several places. It is essential that
they are kept in sync, so mark them with the same string for easy
grepping with an alerting boiler plate.

Signed-off-by: Michael J Gruber <github@grubix.eu>
2018-07-01 12:30:59 +02:00
Oswald Buddenhagen
f82c172d2b fix IMAP UID sequence also in imap_find_new_msgs()
use just * instead of the rather nonsensical *:* (which davmail happens
to actually barf at).

amends 72be55b0 (and 0a5a8479).
2018-07-01 11:05:21 +02:00
Oswald Buddenhagen
904858365d Merge branch '1.3'
Conflicts:
	configure.ac
2018-04-08 18:17:10 +02:00
Oswald Buddenhagen
5072032939 fix uidvalidity recovery with really long message-id headers
re-using the file name buffer for the headers wasn't such a great idea,
as _POSIX_PATH_MAX is only 256, while RFC2822 permits lines up to 1000
chars. and sure enough, i have a message with a whopping 470-char
message-id header ...
2018-04-08 18:10:21 +02:00
Oswald Buddenhagen
0a5a847932 fix IMAP UID sequence in UIDNEXT determination fallback
use just * instead of the rather nonsensical *:* (which davmail happens
to actually barf at).

amends 72be55b0.
2017-11-18 09:59:34 +01:00
Oswald Buddenhagen
af1acdac97 make more use of equals() 2017-10-15 16:55:23 +02:00
Oswald Buddenhagen
c29eceaeed make map_name() interpret empty strings as "no separator"
empty strings were previously meaningless, and starting with 72c2d695a,
failure to handle them lead to bogus results when the IMAP hierarchy
separator is legitimately empty (when the server genuinely supports none
and none is manually configured). non-null can be asserted more cleanly
than null-or-non-empty, so change the api like that.
incidentally, this also removes the need to work around gcc's bogus
warning in -Os mode.

problem found by "Casper Ti. Vector" <caspervector@gmail.com>
2017-10-15 16:53:27 +02:00
Oswald Buddenhagen
53e8e79488 remove pointless conditional in assignment of ctx->delimiter
amends 72c2d695a.
2017-10-15 16:52:59 +02:00
Oswald Buddenhagen
094af8720c limit -Wmaybe-uninitialized suppression to gcc >= 4.3
apple gcc 4.2 complains about the use of the pragma inside a function.
clang also complains, but because the pragma is entirely unknown to it.

as neither compiler emits the bogus warning in the first place, there is
no point in suppressing it anyway.
2017-10-07 16:03:50 +02:00
Oswald Buddenhagen
5aab050198 don't forward-declare SSL types any more
our current project structure precludes the clash between some indirect
include of ssl.h and our definition of 'S' (or 'M', i don't remember)
that happened on some system, so there is no need to avoid including it
any more.

this avoids complaints from some more picky compilers, as re-defining
typedefs (even to the same thing) is illegal before C11.
2017-10-07 16:03:50 +02:00
Oswald Buddenhagen
cbac8aa75c delete the compat wrapper
it was deprecated in 1.2. until 1.4 gets released, enough time will have
passed for sure.
2017-10-01 15:30:07 +02:00
Oswald Buddenhagen
a5d4a0fe60 make sync records with stray TUID non-fatal
while the situation indicates an internal error, it is harmless in
itself. also, printing some more information may help identify the
problem.
2017-10-01 10:42:00 +02:00
Oswald Buddenhagen
3a2e6b3793 prune SSL 2 support
OpenSSL actually did that a while ago already, so this was dead code.
2017-10-01 10:42:00 +02:00
Oswald Buddenhagen
d2e5134ebd another fix for -Wimplicit-fallthrough (new on master) 2017-10-01 10:40:55 +02:00
Oswald Buddenhagen
1b354fa61a Merge branch '1.2'
Conflicts:
	Makefile.am
	configure.ac
	debian/.gitignore
2017-10-01 10:38:43 +02:00
Alessandro Ghedini
a9feea71fe Fix spelling of error messages 2017-10-01 10:04:31 +02:00
Oswald Buddenhagen
33ee4a4ffe fixes for -Wimplicit-fallthrough 2017-10-01 10:04:31 +02:00
Oswald Buddenhagen
47bdbb4aab enable TLS 1.1 and 1.2 by default
there is no reason not to, and debian even disabled 1.0 globally,
because it's (theoretically) too insecure in some contexts (BEAST
attack).

in the compat wrapper, the UseTLSv1 option has been re-interpreted as
v1.x, to avoid adding new options.
2017-08-19 13:15:19 +02:00
Oswald Buddenhagen
118fdc4f18 Merge branch '1.2'
Conflicts:
	configure.ac
	src/mbsync.1
2017-08-05 21:20:48 +02:00
Oswald Buddenhagen
906dc989e4 mention the need for renaming Maildir files upon move
mu4e config line offered by Ben Maughan <benmaughan@gmail.com>.
2017-08-05 20:24:11 +02:00
Oswald Buddenhagen
ada0ae4b8e fix spurious decompression errors
while that's just bad api, inflate() can return Z_BUF_ERROR during
normal operation.

contrary to the zpipe example and what the documentation implies,
deflate() actually isn't that braindead. add respective comments.

REFMAIL: CALA3aExMjtRL0tAmgUANpDTnn-_HJ0sYkOEXWzoO6DVaiNFUHQ@mail.gmail.com
2017-07-30 18:47:30 +02:00
Oswald Buddenhagen
366ed7d762 improve zlib error reporting
zlib is not exactly thorough about always populating z_stream->msg, so
fall back to the error code if necessary.
2017-07-30 13:47:51 +02:00
Patrick Steinhardt
56515abe94 socket: use next addrinfo if opening socket fails
The `socket_connect_one` function previously did an `exit(1)` when
encountering any errors with opening the socket. This would break
connecting to a host where multiple possible addrinfos are returned,
where the leading addrinfos are in fact impossible to connect to. E.g.
with a kernel configured without support for IPv6, the `getaddrinfo`
call may still return a hint containing an IPv6 address alongside
another hint with an IPv4 address. Creating the socket with the IPv6
address, which will cause an error, lead us to exiting early without
even trying remaining hints.

While one can argue that the user should have compiled without HAVE_IPV6
or used an appropriate DNS configuration, we can do better by simply
skipping over the current addrinfo causing an error. To do so, we split
out a new function `socket_connect_next`, which selects the next
available address info and subsequently calls `socket_connect_one`
again. When no hints remain, `sock_connect_one` will error out at that
point.
2017-06-22 09:23:56 +02:00
Oswald Buddenhagen
1039ee25f7 fix build without BDB
amends 83ebe902.

REFMAIL: CAHyO48z0DcoFPC8rCNAL38oxVQtZNKifVd-NEF3sp1EfR-GgxQ@mail.gmail.com
2017-05-14 09:44:43 +02:00
Oswald Buddenhagen
bb632d1cd0 make UIDs unsigned
complies with the IMAP spec, thus removing the (not really) arbitrary
limitation to INT_MAX for UIDs.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
a0961d6505 delay assignment of TUID when propagating messages
go back to assigning TUIDs only right before actually propagating them.
this avoids spurious "TUID lost" warnings.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
bd5fb6fff3 move away from magic UIDs in the sync state
the only legitimate "deviant" UID is zero, meaning "no message". this
can be futher qualified by additional flags in the sync record, rather
than using magic values for the UID. in fact, the zero UID (so far
meaning only "expunged") was already optionally qualifed with "expired".

as a side effect, driver->store_msg() now returns 0 instead of -2 for
unknown UIDs. this was a hack to avoid translating the value later
on, but it made the api horrible, and now it's superflous in the first
place.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
4ffe149666 split off ephemeral sync record state to a separate member
this allows us to simplify logging of expiration, as we now can just log
the entire persistent state instead of fiddling with bits.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
efd72b85cc autotest: implement much more thorough resumption verification
the test will now make a test run for every journaled step, both right
before and right after the logging.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
7ce658d14c autotest: pre-assign all UIDs of the test messages
this ensures stable results when the boxes are used with different
OPEN_FLAGS (which will happen in a subsequent commit), at the negligible
cost of removing the implicit test of the maildir driver's ability to
enumerate new messages.
2017-04-22 11:26:12 +02:00
Oswald Buddenhagen
4cc5ad5a1a introduce driver call debugging
do that by wrapping the actual stores into proxies.

the proxy driver's code is auto-generated from function templates, some
parameters, and the declarations of the driver functions themselves.
attempts to do it with CPP macros turned out to be a nightmare.
2017-04-22 11:26:11 +02:00
Oswald Buddenhagen
bbe4567bce let driver_t::openbox_box() return the UID validity
... and make 'uidvalidity' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
48ad58b9a3 use a #define for invalid UIDVALIDITY 2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
8d4918affd introduce get_uidnext() driver callback
... and make 'uidnext' private to the imap driver.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
683e581340 let driver_t::find_new_msgs() return the list of messages
consistently with driver_t::load_box().
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
74e9368121 let driver_t::load_box() return the list of messages
... and make 'msgs', 'count', and 'recent' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
6e32b88f3d let driver_t::list_store() return the list of boxes
... and make 'boxes' and 'listed' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
8b7d3792e4 factor out transform_refcounted_msg_response()
the missing cross of transform_refcounted_box_response() and
transform_msg_response().
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
39247197f9 make struct imap_cmd_refcounted_state "abstract"
take the callback out of it, so it can be individualized.
so far, this only increases code size ...
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
96b5ae8360 don't mess with the driver's mailbox list from outside
the api specifies that the list remains owned by the driver, so poking
around in it is ugly and risky.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
c886f71054 make driver_t::prepare_load_box() return the final options
... and make 'opts' private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
83ebe9022d introduce get_box_path() driver callback
... and make 'path' private to the maildir driver.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
d624c9af5d make set_bad_callback() a proper driver_t entry
... and make the pointers private to the drivers.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
f46cf8c887 provide a proper getter callback for driver capabilities
that way driver_t contains only callbacks.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
d54809e268 prepend "get_" to getters in driver_t
this makes it callbacks consistently start with a verb.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
36666f7e52 rewrite tracking of highest expired UID
so far, we tracked the slave side, and calculated the master side on the
fly. that complicated things unnecessarily.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
677accfd84 streamline syncing of old entries
the order of the conditionals was purely historical (pre 4ec56f8cf, anno
2005) and hard to follow, as were the comments.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
990c8a1404 sort uid exception list in a smarter place
do it closer to where it is populated. that way the debug output is
sorted, and we don't sort the list if it's known to be empty.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
72be55b0e3 add fallbacks for determining UIDNEXT
if the server sends no UIDNEXT, do an initial FETCH to query the UID of
the last message.

same if the server sends no APPENDUID.

this allows us to remove the arbitrary limitation of the UID range to
INT_MAX, at the cost of additional round-trips.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
416ced25dd fix repeated listing of same Store with different flags
multiple Channels can call driver_t::list_store() with different LIST_*
flags. assuming the flags are actually taken into consideration, using a
single boolean 'listed' flag to track whether the Store still needs to
be listed obviously wouldn't cut it - if INBOX does not live right under
Path and the Channels used entirely disjoint Patterns (say, * and
INBOX*), the second Channel in a single run (probably a Group) would
fail to match anything.

to fix this, make store_t::listed more granular. this also requires
moving its handling back into the drivers (thus reverting c66afdc0),
because the actually performed queries and their possible implicit
results are driver-specific.

note that this slightly pessimizes some cases - e.g., an IMAP Store with
Path "" will now list the entire namespace even if there is only one
Channel with Pattern "INBOX*" (because a hypothetical Pattern "*" would
also include INBOX*, and the queries are kept disjoint to avoid the need
for de-duplication). this isn't expected to be a problem, as listing
mailboxes is generally cheap.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
b9505301cc factor out listing Maildir++ Stores
Maildir++ is sufficiently different from the other SubFolder styles to
justify a separate function; the resulting code duplication is minimal,
but the separated functions are a lot clearer.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
9eaa97923a fix exclusion of bogus "INBOX.*" folders in Maildir++
this also adds code which avoids that the message about excluding the
mailbox is printed multiple times - this could happen with Maildir++, as
the hierarchy is flattened.

amends 0f24ca31b.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
2d1cfc2c7f make "Patterns *" match INBOX* in Maildir++ Stores
this also has the side effect that we won't fail to include INBOX itself
when Inbox is nested under Path when using other SubFolder styles
(regression introduced with Maildir++ support in 0f24ca31b).

REFMAIL: 1489492069.2148187.910409864.7727F9FC@webmail.messagingengine.com
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
763cb8341f make help screen print some more compile time options 2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
28d52b2b18 de-obfuscate cmd_sendable()
split the monster conditional and add comments.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
0aa4c628df add comments 2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
34993fbca6 fix sync resumption with aborted entries
we need a separate log entry type which does proper mmaxxuid tracking.

while moving code around, this also removes a redundant debug statement.

amends b1842617.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
887b2205ff remove nonsensical statement from journal replay of aborted entries
at this stage, entries cannot possibly have messages assigned to them,
so trying to unlink them makes no sense.

amends b1842617.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
5c2ce59217 fix sync resumption with re-newed messages
the UID of the entries needs to be bumped from -1 to -2, as otherwise
the resumed run would see a TUID in a sync entry which may not have one.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
7c466fc3e7 don't emit redundant flag updates for re-newed messages 2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
1ea2e69aa7 fix maxuid tracking
newmaxuid represents the highest UID for which a sync entry was created,
while maxuid represents the end of the range which is guaranteed to have
been propagated. that means that the former needs to be instantly
incremented (and logged), while the latter must not be touched until the
entire new message sync completes. this matters particularly in the case
of resuming an interrupted run, where sync entry creation must resume
exactly where it left off, while loading the box must use the old limit
to ensure that all messages are available for actual propagation.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
6705604c4a de-duplicate journal replay somewhat
we've been using indices to separate master/slave state for a long time,
so there is no point in using pairs of matching brackets to signify the
side in the journal. instead, use somewhat descriptive letters (S[een],
F[ind], T[rashed]) and the index itself.
2017-04-02 17:12:50 +02:00
Oswald Buddenhagen
af4b8896f0 use typedefs for structs more
makes the code more compact (and consistent, as typedefs were already
used in some places).
2017-04-02 15:56:30 +02:00
Oswald Buddenhagen
c3350753b0 factor out jFprintf() 2017-04-02 15:24:03 +02:00
Oswald Buddenhagen
1fdf793a3f fix signedness of 'nex' variables
they are derived from srec->status, which is unsigned. for not
understood reasons, the compiler complains only after extending status
to a full unsigned int.

on the way, localize the declarations.
2017-04-02 12:16:57 +02:00
Oswald Buddenhagen
1e939bafd8 don't use strncpy() to copy TUIDs
latest since 77acc268, the code prior to these statements ensures that
the full length is available, so just use memcpy(). the code for
comparing TUIDs uses memcmp() anyway.
2017-04-02 12:16:57 +02:00
Oswald Buddenhagen
d754608f55 autotest: improve valgrind integration
introduce recognition of $USE_VALGRIND to run all mbsync invocations
through valgrind.

this also removes the seemingly purposeless --log-fd=3 indirection.
2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
f29bed526b autotest: write logs to files
they can get long, and having actual files also helps with diffing.
2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
991e809c38 autotest: factor out readfile() function 2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
2da7951fe0 autotest: de-duplicate error reporting paths 2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
373abcef02 autotest: print consistent information for journal replay failures 2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
ca43c57e85 autotest: don't print expected result if the mbsync run itself fails
there isn't an actual result to compare with anyway.
2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
b4d1a05365 autotest: de-duplicate: use print*() in show*()
this actually adds some (minimal) transformation overhead, but it makes
the relationship clearer.
2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
8aed94420f autotest: determine path of mbsync prior to chdir()
that allows tmp/ to be a symlink to a ramdisk.
2017-04-01 20:32:36 +02:00
Oswald Buddenhagen
71ced65fcc Merge remote-tracking branch 'origin/1.2'
Conflicts:
	src/sync.c
2017-04-01 20:31:51 +02:00
Oswald Buddenhagen
62808c9003 autotest: use warnings 2017-04-01 20:25:58 +02:00
Oswald Buddenhagen
b45e711da5 autotest: remove stray close() call from printstate() 2017-04-01 20:25:53 +02:00
Oswald Buddenhagen
f934e995d6 don't populate sync record map with invalid UIDs
this would obviously just bloat the hash with nonsense, slowing down the
actual lookup later.
2017-03-14 11:36:25 +01:00
Oswald Buddenhagen
f62b3c7be9 fix mislabeling of test 2017-02-15 17:44:35 +01:00
Oswald Buddenhagen
3ebb066aba make -DN print also the sent data 2017-02-15 17:30:15 +01:00
Oswald Buddenhagen
2457b2baa3 don't arbitrarily limit UIDs to a billion, part 2
imap_find_new_msgs() had the same fixed limit as imap_load_box().

amends 815822d8.
2017-02-15 17:25:59 +01:00
Oswald Buddenhagen
77acc26812 implement Message-Id based UIDVALIDITY recovery 2017-01-21 12:09:01 +01:00
Oswald Buddenhagen
f9fe75602e don't fetch message size unless necessary
when syncing flags but not re-newing non-fetched messages, there is no
need to query the message size for all messages, as the old ones are
queried only for their flags.
2017-01-21 11:41:12 +01:00
Oswald Buddenhagen
ec4b21535f some reshuffling in maildir_scan() for clarity 2017-01-21 11:41:12 +01:00
Oswald Buddenhagen
3d64f16702 make fetching of partial UID ranges more flexible
instead of a single hard-coded branch, use a generic method to split
ranges as needed.

this is of course entirely over-engineered as of now, but subsequent
commits will make good use of it.
2017-01-21 11:41:12 +01:00
Oswald Buddenhagen
1d3b36f89e factor out app_cr 2017-01-17 22:14:07 +01:00
Oswald Buddenhagen
3dffd68825 factor out copy_msg_convert() 2017-01-17 22:08:49 +01:00
Oswald Buddenhagen
951b7e77f8 factor out copy_msg_bytes() 2017-01-15 13:25:46 +01:00
Oswald Buddenhagen
509a191474 make more use of strnlen()
includes adding the so far superfluous prototype for the replacemnt to
common.h.
2016-12-29 15:18:39 +01:00
Oswald Buddenhagen
3eea668052 make use of memchr() in strnlen() replacement after all
turns out the comment advising against it was bogus - unlike for
memcmp(), the standard does indeed prescribe that the memchr()
implementation may not read past the first occurrence of the searched
char.
2016-12-29 15:17:04 +01:00
Oswald Buddenhagen
357dd51405 adjust return type of strnlen() replacement
it doesn't really matter, but it's nicer to stay consistent with the
official prototype.
2016-12-29 15:17:03 +01:00
Oswald Buddenhagen
67f4aeff1f standardize on 'int' for message sizes
that's what the sources already assumed anyway. size_t is total
overkill, as No Email Ever (TM) will exceed 2GiB.

this also fixes a harmless format string warning in 32 bit builds.
2016-12-29 14:10:35 +01:00
Oswald Buddenhagen
0c36655201 print actually read TUID in debug message 2016-12-26 16:20:27 +01:00
Oswald Buddenhagen
1330f43034 null-terminate lines read from state file & journal
makes the subsequent code less convoluted.
2016-12-26 16:20:27 +01:00
Oswald Buddenhagen
879eb623be mark string_list_t as packed
otherwise we'll regularly over-allocate due to the struct's stride.
2016-12-18 22:03:51 +01:00
Oswald Buddenhagen
4db64967c9 make more use of shifted_bit()
technically, this introduces a redundant AND, but the compiler is smart
enough to prove that (((A & M) ^ B) & M) == ((A ^ B) & M).
2016-12-18 22:03:51 +01:00