Commit Graph

294 Commits

Author SHA1 Message Date
Oswald Buddenhagen
c83330ffe8 don't unnecessarily re-initialize some members of imap_store
... when recycling server connections.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
cfaa4848dd actually implement imap_commit_cmds()
delay reporting success of STORE FLAGS until a subsequent CHECK
succeeds.

this fixes (inverse flag change propagation) and (deletes not being
propagated) after an interruption due to prematurely logged flag
updates.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
70bad66129 create placeholders for messages over MaxSize
this is vastly more useful than just omitting the messages with no
indication at all.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
0e5046e14a add/fix/de-duplicate comments 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
395f802500 fix loading of some messages' sizes in some partial sync scenarios
we need to pass a different "boundary" UID to driver_t::load_box() for
every OPEN_* flag that queries a partial range:
- OPEN_FIND refers to messages newer than all we know about
- OPEN_OLD_IDS refers to messages which are paired
- OPEN_{OLD,NEW}_SIZE refers to messages (not) above the committed
  boundary of already propagated messages

we treated the 3rd like the 2nd, which was just wrong - the actual
boundary may be lower or higher, so we'd produce wrong results when
MaxSize was set and only one of New and ReNew was requested.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
198ca65b6e add option to get password from macOS Keychain
this is better than using PassCmd, as it allows the keychain manager to
identify the calling process and therefore use a selective whitelist.

unlike in the now removed example, we use an "internet password" for the
imap protocol, rather than a "generic password" - this seems more
appropriate.

based on a patch by Oliver Runge <oliver.runge@gmail.com>
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
03b15dbdd3 add ability to script IMAP user query
It was already possible to retrieve passwords from arbitrary commands.
But this goes only half the way to allowing automated derivation of
login credentials, as some environments may also have different user
names based on the system. Therefore, add the UserCmd option to
complement PassCmd.

Based on a patch series by Patrick Steinhardt <ps@pks.im>
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
503478533c de-duplicate FETCH response data item traversal somewhat 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
8acf56b311 complain about malformed item names in FETCH responses 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
47b477b3fb re-nest parse_fetch_rsp()
prefer early exits over else branches, which is easier to follow.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
81c4bfeefa extract parse_fetched_flags() from parse_fetch_rsp() 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
d4ead05a02 extract parse_fetched_header() from parse_fetch_rsp() 2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
67ea5bea7f handle bogus IMAP FETCH responses more robustly
don't use assert()s when the error condition can stem not only from
errors in mbsync's logic, but also from the IMAP stream being corrupted.

amends 72be55b0e.

REFMAIL: 20191021233411.55ctuvslkfqf2pna@koblih.localdomain
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
a5a8783ea3 sanitize error handling in IMAP FETCH response processing
abort on actual error conditions (protocol errors) and downgrade the
rest to warnings.

REFMAIL: 20191102164509.dxayakg3hrmozjnm@carbon
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
b91dd5b3bc centralize disposal of parsed IMAP lists
makes the code less cluttered, and it's harder to introduce leaks.

this has the hypothetical disadvantage that due to freeing being
delayed, the peak memory usage would rise significantly if we chained to
another parse_list() call which produces a big list while already
holding a big list, but that isn't the case anywhere.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
7af7354dbc fully decompose NAMESPACE response early on
that way the code becomes clearer, and we don't keep useless nodes in
memory.
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
6fd4e8de24 don't store 'shared' and 'other' namespaces
they are never used anyway, and aren't going to be (because configuring
that would be more annoying than just specifying Path manually).
2020-08-04 17:16:03 +02:00
Oswald Buddenhagen
64e5f07ad3 consistently use NULL for null pointers
makes the code noisier, but also somewhat more expressive.
2020-08-04 17:16:01 +02:00
Oswald Buddenhagen
e2d3b4d55b fix lots of sign conversion warnings
... by making a lot of objects unsigned, and some signed.
casts which lose precision and change the sign in one go (ssize_t and
time_t to uint on LP64) are made explicit as well.
2020-08-04 17:15:39 +02:00
Oswald Buddenhagen
cc176df2c3 make some narrowing of integers explicit
this does specifically *not* cover about a bazillion warnings about
size_t being shrunk to uint - these make no sense given the expected
data set size.
2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
5c2e8d3e14 make more objects static 2020-08-04 17:14:55 +02:00
Oswald Buddenhagen
71d7d3e6df add some ATTR_* (mostly)
mostly ATTR_PRINTFLIKE(*, 0) for functions with a va_list argument.

also, one ATTR_NORETURN and one ATTR_UNUSED, both on functions.

also, an explicit suppression for a format string stored in a variable.
2020-08-04 17:13:56 +02:00
Oswald Buddenhagen
234becf530 remove support for SSLv3
it's insecure and default builds of openssl don't include it any more.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
d09f988c70 add support for (disabling) TLS v1.3
this is actually potentially counterproductive, as people who have set
SSLVersions and fail to adjust it will _lose_ tls 1.3 support. however,
without the option being there, people (incorrectly) believe that tls
1.3 is not supported.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
990cc112f1 Add option to use IMAP LSUB instead of LIST
Based on patch by Cedric Ware <cedric.ware__bml@normalesup.org>
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
e9407cc1f7 IMAP: reject unqualified non-uppercased INBOX spellings
otherwise the server would interpret it as INBOX contrary to our
expectations, which might lead to moderately surprising effects.

if you really want to sync your ~/maildir/inbox to the IMAP INBOX,
specify it as the Maildir Store's Inbox.
2020-08-04 14:49:58 +02:00
Jaroslav Suchanek
07cb422cbb Add support for specifying cipher string used for ssl connection
Some distributions (e.g. Fedora) added support for system wide crypto
policies. This is supported in most common crypto libraries including
OpenSSL. Applications can override this policy using their own cipher
string. This commit adds support for specifying the cipher string in
the mbsync configuration.

For example, to exclude Diffie-Hellman, the user can specify
  CipherString "DEFAULT:!DH"
in the IMAP Account's configuration.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
e7bc402d09 rename socket_expect_read() => socket_expect_activity()
... to better reflect its (mostly new) function.
2020-08-04 14:49:58 +02:00
Oswald Buddenhagen
2f3cb5f481 fix signedness issues surrounding UIDs
amends bb632d1c.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
27a1935361 fix strftime() format string warning properly
the workaround for -Wformat triggered -Wformat-nonliteral in turn.
so instead go back to using pragmas and add a proper gcc version check.

this also works with clang - mostly for qt-creator's code model, which
is clang-based.

amends/reverts 55e65147.
2020-08-04 14:49:57 +02:00
Oswald Buddenhagen
5fee222f84 Merge branch '1.3' 2020-08-04 14:49:27 +02:00
Oswald Buddenhagen
542e38dd49 fix re-using server connections for new stores
we failed to reset the box list pointer after freeing it, which would
lead to a crash.
we also failed to reset the listing status, which would lead to
malfunction if we hadn't already crashed.

this inlines imap_cleanup_store(), as there isn't much value in keeping
it. the message list is already freed when disowning the store anyway.
2020-08-04 14:42:42 +02:00
Oswald Buddenhagen
3651c30296 increase PassCmd output buffer even more
apparently, some XOAUTH2 tokens are at 2.4K already, so make it 8K to be
*really* safe for a while.

REFMAIL: <20200716000515.GA2111668@lysator.liu.se>
2020-08-04 14:33:00 +02:00
Oswald Buddenhagen
cab14608ca Merge branch '1.3' 2020-07-08 12:51:20 +02:00
Oswald Buddenhagen
04fc586e75 handle case-insensitivity of IMAP's INBOX
this is relevant only when listing an IMAP Store's contents, as that's
the only place where we aren't imposing the spelling ourselves.

we need to be careful not to treat our own canonical (prefix-stripped
and always slash-delimited) box names like that; codify that in
comments.

this reveals that commit 6f2160f1 may be deemed to have been incorrect -
the TODO item was ambiguous, and could quite possibly have meant this
fix. unsurprisingly, 380ccdd4 re-introduced it with more explicit
wording.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
94022a6752 catch server's failure to FETCH *
the query is untypical enough to have caused problems with davmail (when
we still used *:*) and mailo.com (until it got fixed), so better check
that the result (not) returned by the server makes sense.
2020-07-08 11:14:02 +02:00
Oswald Buddenhagen
93fb3c29c6 fix UIDNEXT error message 2020-07-08 10:50:36 +02:00
Oswald Buddenhagen
e565d08246 don't try to propagate flags the target store does not support
$Forwarded is not standard, so it will most likely fail with mailboxes
that do not support keywords.

amends c4d7f018.
2020-01-08 18:22:48 +01:00
Oswald Buddenhagen
462fed556a Merge branch '1.3' 2019-10-03 20:17:54 +02:00
Dmitry Torokhov
bf14798700 Bump up PassCmd buffer size to 2KiB
While ordinary passwords are rarely longer than 80 bytes, XOAUTH2 tokens
easily exceed this limit. Let's bump it up to 2K to be really safe.
2019-07-19 12:58:04 +02:00
Oswald Buddenhagen
fbc432aace fix parsing of NIL hierarchy delimiters in IMAP LIST responses
a server which does not support hierarchical mailboxes (e.g., seznam.cz
as of oct 2018) can legitimately send NIL (rather than an empty string).
2019-05-28 17:27:09 +02:00
Oswald Buddenhagen
702b6ec4a1 fix spacing in "SASL mechanism(s) not available" error message
amends fdb03b91.
2019-03-10 11:30:05 +01:00
Klemens Nanni
d61f462039 Fix CopyArrivalDate on platforms without glibc
strptime(3)'s "%d" day of the month conversion specifier does not accept
leading blanks in case of single digit numbers.  "%e" does that.

While implementation details and differences between the two
day-of-month conversion specifiers vary, none of the major libcs
(incl. OpenBSD, FreeBSD, Illumos, musl) consume a leading blank for "%d"
except glibc, which consumes any number of spaces like in the "%e" case.

Using "%e" ensures that date strings like " 4-Mar-2018 16:49:25 -0500"
are successfully parsed by all major implementations in compliance to
X/Open Portability Guide Issue 4, Version 2 ("XPG4.2").  musl is now the
only one that still treats "%d" and "%e" without stripping any space.

Issue analysed and reported by Evan Silberman <evan@jklol.net> who found
mbsync 1.3.0 on OpenBSD 6.4 to fail with `CopyArrivalDate' set when
syncing mails with the above mentioned timestamp.

See https://marc.info/?l=openbsd-tech&m=155044284526535 for details.
2019-02-20 23:02:02 +01:00
Gergely Risko
a8f6eebdd9 Work around useless SASL warnings
Ater sasl_client_step() is called and the Cyrus SASL library forwards
it to the client plugin, if the result value is OK (authentication
succeeded), the clientout is filled out to be an empty string, even if
the client plugin wanted to return NULL.

To avoid that mbsync complains at this point, check the returned length
instead of the pointer.
2019-02-05 11:29:13 +01:00
Oswald Buddenhagen
95d18e2778 Merge branch '1.3' 2018-11-27 00:51:03 +01:00
Oswald Buddenhagen
acfa3a2bbc sort messages from UID FETCH request
turns out that some IMAP servers (e.g., poczta.o2.pl) do not return
messages in ascending UID order in response to a UID FETCH request,
which makes the driver violate the API contract.

counter this by sorting the messages. this also addresses the
long-standing (but hypothetical) issue that parallel UID FETCH requests
could be handled out-of-order and thus also lead to mixed up results.

based on patch by Marcin Niestroj <macius1990w@gmail.com>.
2018-11-24 14:36:31 +01:00
Oswald Buddenhagen
f7cec3064d bump IMAP command buffer size to 4KiB
while only 1KiB is required by the IMAP spec, AUTHENTICATE GSSAPI with
Kerberos requires about 1700 bytes.
accomodate that, plus some reserve.

fix suggested by Tollef Fog Heen <tfheen@err.no> via Debian BTS.
2018-09-09 13:43:53 +02:00
Oswald Buddenhagen
37feeddbfb Merge branch '1.3' 2018-07-01 13:26:10 +02:00
Oswald Buddenhagen
f698f16967 fix type of 'port' and check its range in config reader 2018-07-01 13:25:16 +02:00
Michael J Gruber
c4d7f0189c implement Forwarded flag
maildir supports a 'P' flag which denotes the fact that a message has
been 'passed' on (forwarded, bounced). notmuch syncs this to the
'passed' tag.

Per https://tools.ietf.org/html/rfc5788, IMAP has a user-defined flag
(keyword) '$Forwarded' that is supported by many servers and clients
these days. (Technically, one should check for '$Forwarded' in the
server response.)

Restructure mbsync's flag parser to accept keywords (flags starting with
'$') but still bail out on unknown system flags (flags starting with '\').
Support '$Forwarded' as a first keyword since it maps to maildir's 'P'
and needs to be sorted in between the system flags.

Signed-off-by: Michael J Gruber <github@grubix.eu>
2018-07-01 12:36:28 +02:00