Commit Graph

389 Commits

Author SHA1 Message Date
Oswald Buddenhagen
55e65147df fix compilation with older gcc versions
the warning suppression pragma within function scope is apparently a new
thing.
as i don't want to disable the check for the entire function (even if
this currently would make no difference), just use a wrapper function
to suppress the format string check.
2013-12-01 13:36:26 +01:00
Oswald Buddenhagen
32def5dc0a add/fix comments and improve debug messages 2013-12-01 13:36:26 +01:00
Oswald Buddenhagen
a9a331c98a simplify condition
... and document the cases.
2013-12-01 13:35:02 +01:00
Oswald Buddenhagen
03f8bfdfb2 micro-optimization/-clarification 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
00076a6971 move initializations for clarity 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
61ef099cd5 MaxMessages: make condition exactly symmetrical to condition below 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
080740f867 rewrite condition for readability and consistency 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
b10fd0c21c remove assumption about value of M constant 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
a893cba483 fix enum abuse
amends 9c86ec344.

S_FIND was for the sync record status field. it has no business in the
sync vars status fields. its value coincided with ST_SELECTED, which
luckily only means that we always tried to match up TUIDs even if there
was nothing to do.

the need for TUID matching arises in two mostly independent
circumstances, so add two separate flags ST_FIND_{OLD,NEW}.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
9a0e65f899 create unseen messages in /new/
seen messages still go to /cur/.
this is consistent with the actual maildir driver.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
0a684bd933 be a bit more verbose 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
394aca03a2 properly handle unexpected exit while replaying journal 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
07377cb753 ensure that the journal replay pass really does nothing 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
100f9487f4 sort messages by serial number instead of UID in box dumper
the input data is sorted that way, so it's easier to compare.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
2568459a7b fix error message in sync state dumper 2013-12-01 13:35:00 +01:00
Oswald Buddenhagen
b570c17766 deal with messages disappearing between being listed and fetched 2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
17c4748dfa support backslash-escaping in the config file
note that no attempt is made at making this work in the compat wrapper.
2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
2213d6976c support backslashes and quotes in quoted IMAP strings
the RFCs require it - well hidden in the BNF at the bottom.

patch somewhat inspired by "guns" <self@sungpae.com>.
2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
c0bf867669 make next_arg() more readable & efficient 2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
3ceb553102 IPv6 support
inspired by a patch by "Todd T. Fries" <todd@fries.net>.
2013-09-01 17:39:07 +02:00
Oswald Buddenhagen
4a39cae8c4 support multi-homed servers 2013-09-01 17:39:07 +02:00
Oswald Buddenhagen
0b59ee0df3 support multi-character path separators
this applies to both the IMAP PathDelimiter (which is needed by Lotus
Domino), as well as the Flatten-ed separators.
2013-08-11 10:20:02 +02:00
Oswald Buddenhagen
1b67c49965 CHECK before FETCH after STORE
m$ exchange does not seem to update the index in time otherwise.
2013-08-11 10:20:02 +02:00
Oswald Buddenhagen
eb1f10762f added sync support for the arrival date of messages
initial patch by Marc Hoersken <info@marc-hoersken.de>
2013-08-03 18:54:34 +02:00
Oswald Buddenhagen
6577bf3e61 warn if we cannot find some messages by TUID 2013-07-27 20:18:20 +02:00
Oswald Buddenhagen
1847a4e12d make better use of ATTR_UNUSED 2013-07-27 18:44:26 +02:00
Oswald Buddenhagen
6dfccb76a5 be somewhat stricter about the LIST response syntax
the first token *must* be a list.
2013-07-27 18:40:19 +02:00
Oswald Buddenhagen
a0dc37339e allow the mailbox names in LIST responses to be literals
Lotus Domino seems to send them like that.
2013-07-27 18:40:16 +02:00
Oswald Buddenhagen
fc77feacc5 make parse_list() callback based
this allows us to parse IMAP literals ({}) in every list.
2013-07-27 18:39:39 +02:00
Oswald Buddenhagen
bf049d6466 add PassCmd option to query IMAP password dynamically
inspired by patches by
Aurélien Francillon <aurelien.francillon@eurecom.fr>,
Martin Stenberg <martin@gnutiken.se> and
sbfnk@users.sf.net.
2013-07-27 11:31:31 +02:00
Oswald Buddenhagen
5ad83b4e6a don't unnecessarily use continue 2013-07-27 09:34:17 +02:00
Oswald Buddenhagen
e4243debb6 use INT_MAX instead of zero for "no size limit"
this simplifies the actual conditions
2013-07-27 09:34:17 +02:00
Oswald Buddenhagen
ca3a319e60 update copyrights 2013-04-20 16:57:16 +02:00
Oswald Buddenhagen
01358ec8b4 man page fixups 2013-04-13 20:07:16 +02:00
Oswald Buddenhagen
406e967430 don't let wildcards match INBOX, unless it lives under Path
it's counter-intuitive to have '*' match the (always present) INBOX
when the rest of the mailboxes lives in a different namespace.
2013-04-13 19:58:50 +02:00
Oswald Buddenhagen
e7c96f8891 always list INBOX when asked for it
it's there even if it's not there. says IMAP. no need to contradict.
2013-04-13 19:57:41 +02:00
Oswald Buddenhagen
daaf950878 split maildir_list_part()
the boolean argument switched two entirely separate functions.
2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
167de3e438 remove some temporaries in maildir_list_part() 2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
842aa402c3 fix CRAM-MD5 authentication
the decoded challenge may be padded, so we really need to use strlen()
rather than just the decoded length.
2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
e07de2a336 more consistency in char signedness 2013-04-13 10:54:50 +02:00
Felipe Contreras
aad7f903ec maildir: fix trash path double-free
It should be freed at the very end.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2013-04-07 18:32:52 +02:00
Oswald Buddenhagen
89c81e382e discourage use of MapInbox 2013-04-07 16:57:45 +02:00
Oswald Buddenhagen
4ab55dc468 don't try to fetch status of minus one message
this would happen if we were trying to find newly pushed messages, but
none actually arrived.
as imap's ranges are not ordered, this would actually fetch one message.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
9261897629 don't record newuid in the sync state
this value is only ever used to find just pushed messages by TUID, so we
can simply use the UIDNEXT value from before we started pushing - and of
course, we need to record that in the journal. it makes no sense to log
the new value after completing a search, as there won't be a next search
before we push the next messages.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
96be183acb rename sync_vars_t::uidnext => newuid & fix comment
the purpose of this variable is to hold the UIDNEXT value from before
we started pushing new messages, i.e., the minimal uid we can expect
them to have.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
15d57b95b7 make paths relative to CWD, after all
the test suite actually relies on it. it would be possible to adjust it,
but there is not much reason to make paths relative to HOME (as we
support convenient tilde expansion). so use the least invasive approach,
which is simply the old behavior. adjust the documentation accordingly.

This reverts commit da5ce5d8f4.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
7ba7be111e improve socket error reporting
always use getsockopt() to query the meaning of POLLERR, rather than
reporting "Unidentified socket error".
this is unlikely to have any effect when using select(), as that one
pretty much never signals exceptional conditions.
2013-03-29 18:25:39 +01:00
Oswald Buddenhagen
2ef6dc8a90 factor out socket_connect_failed() 2013-03-29 18:25:39 +01:00
Oswald Buddenhagen
37a28d8133 improve socket connect() error reporting with poll()
turns out that poll() may (and on linux does) signal POLLERR on
connection failure. this is unlike select(), which is specified to
signal write readiness in every case.
consequently, check whether we are connecting before checking for
POLLERR.
2013-03-29 18:24:32 +01:00
Oswald Buddenhagen
d1900941f4 introduce -DC option to only install a crash handler 2013-03-29 16:54:45 +01:00
Oswald Buddenhagen
b142778e56 match flag names in man page
"Full" is an alias for "All", but let's stick to one.
2013-03-24 18:42:57 +01:00
Oswald Buddenhagen
da5ce5d8f4 make path expansion match docu: paths are relative to ~
the current behavior of being relative to the current directory sort of
makes no sense, and contradicts the docu.
2013-03-24 18:42:57 +01:00
Oswald Buddenhagen
312f4be4b2 disable SSLv2 by default in the wrapper as well 2013-03-24 11:14:34 +01:00
Oswald Buddenhagen
89add4f330 downcast time_t to long for printing
time_t may be long long. to keep the sprintf format strings simple, just
downcast - this is not going to be a problem for the next 30 years, and
until then long will be 64-bit everywhere anyway.

suggested 3.5 years ago by Antoine Reilles <tonio@NetBSD.org>.
2013-03-24 11:14:27 +01:00
Oswald Buddenhagen
fbfcfea5dc fix cram-md5 authentication
we need to send a newline after the response for imap to grok it.
2013-03-23 10:38:27 +01:00
Oswald Buddenhagen
3363ad0f11 fix crashes in imap_open_store() error paths
it's not a good idea to invoke imap_open_store_bail() twice, either ...
2013-03-23 10:34:51 +01:00
Oswald Buddenhagen
a66034b23a fix crash in ssl connection error path
not a good idea to invoke the callback twice ...
2013-03-17 19:49:53 +01:00
Oswald Buddenhagen
acb1c870b4 rewrite SSL certificate verification. again.
leave all the hard work to OpenSSL. this has several consequences:
- certificate chain validation actually works instead of throwing
  around error 20
- the interactive approval is gone. i don't expect it to be useful
  anyway, as mbsync is mostly a batch tool
- the code is much shorter
2013-03-17 19:49:03 +01:00
Oswald Buddenhagen
8310cf78ac fix CVE-2013-0289: add SSL subject verification
we did not check a valid certificate's subject at all so far.
this is no problem if the certificate file contains only exactly the
wanted host's certificate - before revision 04fdf7d1 (dec 2000, < v0.4),
this was even enforced (more or less - if the peer cert had been
signed directly by a root cert, it would be accepted as well).
however, when the file contains root certificates (like the system-wide
certificate file typically does), any host with a valid certificate
could pretend to be the wanted host.
2013-02-17 18:33:04 +01:00
Oswald Buddenhagen
fbba8f1cda add support for (disabling) TLS v1.1 and v1.2 2013-02-10 09:56:33 +01:00
Oswald Buddenhagen
c7ebe2da95 more error checking of IMAP responses
REFMAIL: CA+Tk8fyu-6bwXq=ee2BgcKK_13m9S0RS+-0DhM=_jFqSKCH8aw@mail.gmail.com
2012-10-16 09:37:19 +02:00
Oswald Buddenhagen
8dbb3fe7a9 flush stdout more
to make sure it is timely written and not interleaved with stderr even
when when redirected.
2012-09-22 17:48:09 +02:00
Oswald Buddenhagen
d7eae525bd fix TrashRemoteNew copy direction 2012-09-22 17:35:39 +02:00
Oswald Buddenhagen
c23d251092 consider hierarchy delimiter flattening when deciding what to list
flattened sub-folders of INBOX actually end up in Path, so list that
instead.

REFMAIL: 6c0ecbff0d025387020281c5d2f5e6e8@smallsys.org
2012-09-16 13:06:07 +02:00
Oswald Buddenhagen
66895f9cce try harder to list all necessary boxes
the pattern "INB*" may or may not refer to something in the INBOX. even
just "*" may. so list both the INBOX and the Path in case of
uncertainty.
2012-09-16 12:34:07 +02:00
Oswald Buddenhagen
35851f133b add option to control amount of fsync()ing 2012-09-15 15:28:15 +02:00
Oswald Buddenhagen
49223b2df2 avoid that a system crash can cause messages to be propagated twice
fdatasync() the journal after creating the pair record and recording
the TUID, but before the message propagation actually starts.

all other writes to the journal are not flushed, as they will at worst
cause some unnecessary network traffic without visible effect.
2012-09-15 15:28:15 +02:00
Oswald Buddenhagen
a326bf2f58 avoid that a system crash can lose mails
this fixes two possible failure scenarios:
- if the journal is committed but the mails are not, the missing files
  would be erroneously interpreted as deletions which would be
  propagated
- less seriously, if the mail files' meta data was committed but the
  file contents were not, we would end up with empty files, which would
  have to be re-fetched "behind mbsync's back" (just deleting the files
  would not work - see above)
2012-09-15 15:27:23 +02:00
Oswald Buddenhagen
df6c3b64b7 avoid that a system crash can clobber the sync state file
make sure that the new state is committed to disk before overwriting the
old version - by default meta data is committed first, so we may end up
with no valid state at all otherwise.
2012-09-15 13:25:50 +02:00
Oswald Buddenhagen
bbf98bb165 quote mailbox names written to config file 2012-09-15 11:57:14 +02:00
Oswald Buddenhagen
18936f6696 make more config file errors fatal
we really shouldn't just synchronize despite config parsing errors.
2012-09-15 11:49:24 +02:00
Oswald Buddenhagen
16e5aade3f store config error status in conffile_t object
this makes passing it around more straight-forward
2012-09-15 11:46:42 +02:00
Oswald Buddenhagen
725a122e91 make config parser a bit more careful about quotes
the parsing is more shell-like now:
- quoted and unquoted parts can be mixed in one argument
- the hashmark can be meaningfully quoted
2012-09-15 11:24:57 +02:00
Oswald Buddenhagen
2e07e68630 call fdatasync() after updating .uidvalidity files
they must be flushed before the file system meta data, as otherwise we
may end up with duplicate UIDs after a system crash.
2012-09-09 12:18:14 +02:00
Ben Kibbey
47fe4b7998 Fix certificate verification.
The connection state wasn't getting updated.
2012-09-08 14:26:22 +02:00
Oswald Buddenhagen
0a8f19294c pre-release doc updates 2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
f11504aa07 update copyrights
make the wrapper's help string also mention copyrights pertaining only
to the actual syncer, as this is the only string many people will ever
see.
2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
d4c786823d replace FSF address with something more ... contemporary 2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
4e849196b8 install the config examples to docdir 2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
6d49c343fc use a hash table for message => sync record lookup
this removes the pathological O(<number of sync records> * <number of
new messages>) case at the cost of being a bit more cpu-intensive (but
O(<number of all messages>)) for old messages.
2012-09-01 21:15:53 +02:00
Oswald Buddenhagen
18225344c6 make use of UID EXPUNGE 2012-09-01 21:15:52 +02:00
Oswald Buddenhagen
dfd7516b9a introduce ability to flatten the hierarchy of Stores 2012-09-01 21:15:52 +02:00
Oswald Buddenhagen
2585dd3324 add support for hierarchical mailboxes 2012-09-01 21:15:18 +02:00
Oswald Buddenhagen
4f94197e41 calculate trash box path already in maildir_open_store()
this gives us some cleaner code paths later on, as we can treat the
trash box like a regular mailbox.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
8121224744 ensure that mailbox creation in maildir_store() is limited to trashing
other mailboxes would have been maildir_select()ed already.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
a3f66f8f1d refactor: imap_select2_p2 => imap_refcounted_done_box
soon, we'll use it for something different, too
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
343f16771a don't crash when select() on master fails synchronously
svars->drv[S] would not be initialized yet, so cancel_sync() would
crash.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
28cccf4b35 fix error handling of invalid SyncState *
when we find that the store is incompatible with in-store sync state,
we want to fail the whole channel. however, we must not claim that the
store died, otherwise it won't be disposed of properly.
2012-09-01 21:15:08 +02:00
Oswald Buddenhagen
1bc9c6d9cf reject qualified mailboxes with the magic name INBOX
otherwise we couldn't tell them apart from the real INBOX after
stripping away the Path.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
40f2812a41 suppress bdb complaints about unknown file format
pass DB_TRUNCATE when creating databases. otherwise bdb will complain
about the empty file we pass it (we have to create it upfront to
implement our locking).
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
9bbb02b8fd Revert "fix UIDNEXT handling"
in fact, UIDNEXT (and UIDVALIDITY) null is *not* allowed (see RFC3501
section 9). them popping up nonetheless was a dovecot bug (which would
also confuse dovecot itself).
having it in as a workaround was no good either, as quite some other
code in mbsync assumes that UIDs are not null.

This reverts commit e1fa867 and most of 39006d7.

-REFMAIL: 4CA62BA1.4020104@lemma.co.uk
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
233f563569 deal with concurrent maildir modifications during listing
files may be renamed (due to new -> cur transition or flag changes),
which may lead to two effects if ignored:
- we see both the old and the new name, so we report a spurious
  duplicate UID
- we see neither name, so we report a spurious deletion

as countermeasure, record and compare directory modification times. upon
mismatch, we just start over - as usual.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
3386285205 make maildir uidvalidity change fatal
it's best to give the user a chance to fix it rather than completely
messing up the syncstate by re-enumerating the UIDs.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
a3bd10c04d cleanup maildir error paths
don't try to unlock and close databases and files - this will happen a
moment later anyway, through cancelation or re-selection.
ironically, this plugs a memory leak, because an open main database is
used as a signal to close a temporary db in maildir_scan().
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
04ca97920d fix potential double free
the store may be discarded before we reach maildir_select() again, which
will leave us with a dangling pointer.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
e71ad53b7f plug memory leak in maildir_store_msg() upon failure to acquire UID 2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
9c86ec3442 employ alternative scheme to finding messages by TUID
instead of SEARCHing every single message (which is slow and happens to
be unreliabe with M$ Exchange 2010), just FETCH the new messages from
the mailbox - the ones we just appended will be amongst them.
2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
b4cef554fc clearer debug msg 2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
06b303da88 use ATTR_PRINTFALIKE 2012-09-01 21:15:07 +02:00
Oswald Buddenhagen
7c815538ab fix line wrapping before info messages
unless an info message is explictly marked as a continuation, it must
terminate any pending line (typically the progress information) first.

debug output is not affected, as it is mutually exclusive with info
output, and no debug lines are left unterminated outside clear scopes.
2012-09-01 21:15:07 +02:00