makes the code less cluttered, and it's harder to introduce leaks.
this has the hypothetical disadvantage that due to freeing being
delayed, the peak memory usage would rise significantly if we chained to
another parse_list() call which produces a big list while already
holding a big list, but that isn't the case anywhere.
... by making a lot of objects unsigned, and some signed.
casts which lose precision and change the sign in one go (ssize_t and
time_t to uint on LP64) are made explicit as well.
this does specifically *not* cover about a bazillion warnings about
size_t being shrunk to uint - these make no sense given the expected
data set size.
mostly ATTR_PRINTFLIKE(*, 0) for functions with a va_list argument.
also, one ATTR_NORETURN and one ATTR_UNUSED, both on functions.
also, an explicit suppression for a format string stored in a variable.
this is actually potentially counterproductive, as people who have set
SSLVersions and fail to adjust it will _lose_ tls 1.3 support. however,
without the option being there, people (incorrectly) believe that tls
1.3 is not supported.
otherwise the server would interpret it as INBOX contrary to our
expectations, which might lead to moderately surprising effects.
if you really want to sync your ~/maildir/inbox to the IMAP INBOX,
specify it as the Maildir Store's Inbox.
Some distributions (e.g. Fedora) added support for system wide crypto
policies. This is supported in most common crypto libraries including
OpenSSL. Applications can override this policy using their own cipher
string. This commit adds support for specifying the cipher string in
the mbsync configuration.
For example, to exclude Diffie-Hellman, the user can specify
CipherString "DEFAULT:!DH"
in the IMAP Account's configuration.
the workaround for -Wformat triggered -Wformat-nonliteral in turn.
so instead go back to using pragmas and add a proper gcc version check.
this also works with clang - mostly for qt-creator's code model, which
is clang-based.
amends/reverts 55e65147.
we failed to reset the box list pointer after freeing it, which would
lead to a crash.
we also failed to reset the listing status, which would lead to
malfunction if we hadn't already crashed.
this inlines imap_cleanup_store(), as there isn't much value in keeping
it. the message list is already freed when disowning the store anyway.
this is relevant only when listing an IMAP Store's contents, as that's
the only place where we aren't imposing the spelling ourselves.
we need to be careful not to treat our own canonical (prefix-stripped
and always slash-delimited) box names like that; codify that in
comments.
this reveals that commit 6f2160f1 may be deemed to have been incorrect -
the TODO item was ambiguous, and could quite possibly have meant this
fix. unsurprisingly, 380ccdd4 re-introduced it with more explicit
wording.
the query is untypical enough to have caused problems with davmail (when
we still used *:*) and mailo.com (until it got fixed), so better check
that the result (not) returned by the server makes sense.
strptime(3)'s "%d" day of the month conversion specifier does not accept
leading blanks in case of single digit numbers. "%e" does that.
While implementation details and differences between the two
day-of-month conversion specifiers vary, none of the major libcs
(incl. OpenBSD, FreeBSD, Illumos, musl) consume a leading blank for "%d"
except glibc, which consumes any number of spaces like in the "%e" case.
Using "%e" ensures that date strings like " 4-Mar-2018 16:49:25 -0500"
are successfully parsed by all major implementations in compliance to
X/Open Portability Guide Issue 4, Version 2 ("XPG4.2"). musl is now the
only one that still treats "%d" and "%e" without stripping any space.
Issue analysed and reported by Evan Silberman <evan@jklol.net> who found
mbsync 1.3.0 on OpenBSD 6.4 to fail with `CopyArrivalDate' set when
syncing mails with the above mentioned timestamp.
See https://marc.info/?l=openbsd-tech&m=155044284526535 for details.
Ater sasl_client_step() is called and the Cyrus SASL library forwards
it to the client plugin, if the result value is OK (authentication
succeeded), the clientout is filled out to be an empty string, even if
the client plugin wanted to return NULL.
To avoid that mbsync complains at this point, check the returned length
instead of the pointer.
turns out that some IMAP servers (e.g., poczta.o2.pl) do not return
messages in ascending UID order in response to a UID FETCH request,
which makes the driver violate the API contract.
counter this by sorting the messages. this also addresses the
long-standing (but hypothetical) issue that parallel UID FETCH requests
could be handled out-of-order and thus also lead to mixed up results.
based on patch by Marcin Niestroj <macius1990w@gmail.com>.
while only 1KiB is required by the IMAP spec, AUTHENTICATE GSSAPI with
Kerberos requires about 1700 bytes.
accomodate that, plus some reserve.
fix suggested by Tollef Fog Heen <tfheen@err.no> via Debian BTS.
maildir supports a 'P' flag which denotes the fact that a message has
been 'passed' on (forwarded, bounced). notmuch syncs this to the
'passed' tag.
Per https://tools.ietf.org/html/rfc5788, IMAP has a user-defined flag
(keyword) '$Forwarded' that is supported by many servers and clients
these days. (Technically, one should check for '$Forwarded' in the
server response.)
Restructure mbsync's flag parser to accept keywords (flags starting with
'$') but still bail out on unknown system flags (flags starting with '\').
Support '$Forwarded' as a first keyword since it maps to maildir's 'P'
and needs to be sorted in between the system flags.
Signed-off-by: Michael J Gruber <github@grubix.eu>
Mailbox driver flags are defined in several places. It is essential that
they are kept in sync, so mark them with the same string for easy
grepping with an alerting boiler plate.
Signed-off-by: Michael J Gruber <github@grubix.eu>
there is no reason not to, and debian even disabled 1.0 globally,
because it's (theoretically) too insecure in some contexts (BEAST
attack).
in the compat wrapper, the UseTLSv1 option has been re-interpreted as
v1.x, to avoid adding new options.
the only legitimate "deviant" UID is zero, meaning "no message". this
can be futher qualified by additional flags in the sync record, rather
than using magic values for the UID. in fact, the zero UID (so far
meaning only "expunged") was already optionally qualifed with "expired".
as a side effect, driver->store_msg() now returns 0 instead of -2 for
unknown UIDs. this was a hack to avoid translating the value later
on, but it made the api horrible, and now it's superflous in the first
place.
do that by wrapping the actual stores into proxies.
the proxy driver's code is auto-generated from function templates, some
parameters, and the declarations of the driver functions themselves.
attempts to do it with CPP macros turned out to be a nightmare.
if the server sends no UIDNEXT, do an initial FETCH to query the UID of
the last message.
same if the server sends no APPENDUID.
this allows us to remove the arbitrary limitation of the UID range to
INT_MAX, at the cost of additional round-trips.
multiple Channels can call driver_t::list_store() with different LIST_*
flags. assuming the flags are actually taken into consideration, using a
single boolean 'listed' flag to track whether the Store still needs to
be listed obviously wouldn't cut it - if INBOX does not live right under
Path and the Channels used entirely disjoint Patterns (say, * and
INBOX*), the second Channel in a single run (probably a Group) would
fail to match anything.
to fix this, make store_t::listed more granular. this also requires
moving its handling back into the drivers (thus reverting c66afdc0),
because the actually performed queries and their possible implicit
results are driver-specific.
note that this slightly pessimizes some cases - e.g., an IMAP Store with
Path "" will now list the entire namespace even if there is only one
Channel with Pattern "INBOX*" (because a hypothetical Pattern "*" would
also include INBOX*, and the queries are kept disjoint to avoid the need
for de-duplication). this isn't expected to be a problem, as listing
mailboxes is generally cheap.
latest since 77acc268, the code prior to these statements ensures that
the full length is available, so just use memcpy(). the code for
comparing TUIDs uses memcmp() anyway.
when syncing flags but not re-newing non-fetched messages, there is no
need to query the message size for all messages, as the old ones are
queried only for their flags.
instead of a single hard-coded branch, use a generic method to split
ranges as needed.
this is of course entirely over-engineered as of now, but subsequent
commits will make good use of it.
if AuthMechs includes more than just LOGIN and the server announces any
AUTH= mechanism, we try SASL. but that can still fail to find any
suitable authentication mechanism, and we must not error out in that
case if we are supposed to fall back to LOGIN.
specifically, if AuthMechs included more than just LOGIN (which would be
the case for '*') and the server announced any AUTH= mechanism, we'd
immediately error out upon seeing it, thus failing to actually try
LOGIN.
the number was chosen to make queries more comprehensible when the
server sends no UIDNEXT, but it appears that such insanely large UIDs
actually show up in the wild. so send 32-bit INT_MAX instead.
note that this is again making an assumption: that no server uses
unsigned ints for UIDs. but we can't sent UINT_MAX, as that would break
with servers which use signed ints. also, *we* use signed ints (which is
actually a clear violation of the spec).
it would be possible to special-case the range [1,inf] to 1:*, thus
entirely removing arbitrary limits. however, when the range doesn't
start at 1, we may actually get a single message instead of none due to
the imap uid range limits being unordered. this gets really nasty when
we need to issue multiple queries, as we may list the same message
twice.
a reliable way around this would be issuing a separate query to find the
actual value of UID '*', to make up for the server not sending UIDNEXT
in the first place. this would obviously imply an additional round-trip
per mailbox ...
it is legal for an email system to simply change the case of rfc2822
headers, and at least one imap server apparently does just that.
this would lead to us not finding our own header, which is obviously not
helpful.
REFMAIL: CA+fD2U3hJEszmvwBsXEpTsaWgJ2Dh373mCESM3M0kg3ZwAYjaw@mail.gmail.com
that pattern may very well expand to INBOXNOT, which would naturally
live under Path, so we need to look into the Path. of course, this
actually makes sense only if there *is* a Path, and complaining about
it being absent is backwards.
the idea that this is even possible was based on an incomplete reading
of the imap spec.
however, the infrastructure for supporting multi-char delimiters as such
is retained, as the Flatten option can be used with them.
recycling server connections skips everything up to setting up the
prefix (Path/NAMESPACE). "everything" should obviously include enabling
compression, as that must be done at most once per connection.
any structures may be invalid after callback invocation.
this has the side effect that the socket write callback now returns
void, like all other callbacks do.
the synchronous writing to the socket would have typically invoked the
write callback, which would flush further commands, thus recursing.
we take the easy way out and make it fully asynchronous, i.e., no data
is sent before (re-)entering the event loop.
this also has the effect that socket_write() cannot fail any more, and
any errors will be reported asynchronously. this is consistent with
socket_read(), and produces cleaner code.
this introduces a marginal performance regression: the maildir driver is
synchronous, so all messages (which fit into memory) will be read before
any data is sent. this is not considered relevant.
