Commit Graph

430 Commits

Author SHA1 Message Date
Oswald Buddenhagen
c293acaf24 fix listing of nested maildir mailboxes 2013-12-08 11:12:18 +01:00
Oswald Buddenhagen
cf0f32f800 allow prefixes to Patterns
this makes it possible to "rename" a "namespace" while syncing.
2013-12-08 11:12:18 +01:00
Oswald Buddenhagen
6c6ad9710c less spaghetti 2013-12-08 11:12:18 +01:00
Oswald Buddenhagen
decc33c2cf factor out sync_listed_boxes() 2013-12-08 11:12:18 +01:00
Oswald Buddenhagen
f485d69332 refuse box list overrides if Channel has no Patterns
as the named boxes are the same on both sides, they logically make
sense only when the channel is in that mode anyway, which is the case
when using patterns.
2013-12-08 11:12:17 +01:00
Oswald Buddenhagen
c6f08b8f17 treat manually specified box list the same as one coming from Patterns 2013-12-08 11:12:17 +01:00
Oswald Buddenhagen
540adbb8fd make host resolution error messages more useful in non-ipv6 builds 2013-12-08 11:12:10 +01:00
Oswald Buddenhagen
b6949c64d2 avoid useless delay after creating maildir box
we would see the recent timestamp of the creation and conclude that
something is going on, so we'd wait. this is obviously nonsense.
as we know that a freshly created mailbox is empty, simply skip the
message scan alltogether.
2013-12-08 11:12:10 +01:00
Oswald Buddenhagen
71524cb6b0 reduce FSync option to a boolean
there is no use for Thorough mode any more, so simplify the
configuration.
2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
29a56e2dc4 don't fsync after logging every TUID
as we now don't actually start propagating new messages until all TUIDs
have been generated, it's sufficient to sync just once. this makes it
a cheap operation, so we can do it at SYNC_NORMAL level already.
2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
8d5bd62537 add ExpireUnread option 2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
f586c0bee5 make it possible to specify CopyArrivalDate and MaxMessages globally
sneaky change on the side: the wording of the man page is changed from
"outside any section" to "before any section" to get global options.
this is not entirely true ... the previously existing options behave as
before, while the two newcomers actually affect subsequent channels.
2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
c0ba0c7ecf replace global_* with a channel_conf_t instance
this makes the (growing) list of getopt_helper()'s parameters
manageable. the few wasted bytes are worth it.
2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
1e427f5cd5 do not unnecessarily use bitfields
they don't save much (if any) space in our usage, while they make the
machine code more bloated and slow.
2013-12-08 11:12:09 +01:00
Oswald Buddenhagen
49a32910a7 move handling of new messages after that of old ones
i.e., move it back. whatever the original reason was, it's now gone.

this order is way more natural, which allows us to remove the osrecadd
and S_DONE hacks.
2013-12-01 13:36:28 +01:00
Oswald Buddenhagen
fe3d19b7eb verify idempotence of all sync operations 2013-12-01 13:36:28 +01:00
Oswald Buddenhagen
b1842617f7 make MaxMessages work for new mails as well
this helps enormously on the first sync of a 100k message box with a
limit of 1k messages. it also happens to make the syncing idempotent.

in a few conditionals we now explicitly test for max_messages being
enabled, not smaxxuid != 0, as after the initial fetch with no important
messages smaxxuid is zero, but we still have to skip over 99k messages
in the above case.
2013-12-01 13:36:28 +01:00
Oswald Buddenhagen
d3f6347021 delay propagation of new messages
previous sequence:
  examine & propagate new => examine old => propagate old
new sequence:
  examine new => examine old => propagate new => propagate old

this alone does not buy us much ...
2013-12-01 13:36:28 +01:00
Oswald Buddenhagen
391ec01f28 make message propagation recording less magic
assign the sync record to the source message asap, and later on rely
on a more explicit condition than not doing so.
2013-12-01 13:36:28 +01:00
Oswald Buddenhagen
7f784fd235 log maxuid bumping less aggressively
we can bump the internal variable whereever convenient, but we cannot
log it until we know that all messages were copied, as otherwise we
could miss some new messages after an interruption. with the new
approach, interruption would merely cause some additonal traffic.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
8b76412b0d document message expiration transactions 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
ecb4c7ab07 propagate deletions with other flag changes
less code duplication, more logical order of issued driver commands
(especially after the next commit), and the "side effect" of letting the
message expiration code see those deletions if they are asynchronous.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
273ac899f3 don't delay loading master even if messages were expired
the delay optimized the corner case of previously important but now
expired messages on the slave disappearing, either through an external
expunge or after a journal replay. no point in pessimizing the common
case.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
12676f28da remove cleanup of expired entries during setup of master load
the removed code would only ever trigger if a) we were after a journal
replay or b) something external expunged the expired messages - both are
corner cases not worth the extra code.
however, this means that the syncing code further down now needs to take
care of these zombies.
in the end, the normal cleanup will take care of all expired entries,
new and old.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
9a62521cff micro-optimization/-clarification: swap condition order 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
014d9b9081 make message counting in expiration code less confusing 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
83b834cdfd count unread messages like flagged messages when expiring
that is, don't count them towards the total only below the cut-off
point. making them extend the working set even though they are inside it
is counterintuitive.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
9e186ae88b use post-sync "seen" flag to determine expirability
otherwise it wouldn't be idempotent.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
15216947fb don't protect recent messages from MaxMessages
while maildir has a clearly defined meaning of "recent" and for example
mutt handles it graciously, IMAP's definition is fubared to the point
that some servers (for example gmail) simply refuse to support it.
for symmetry reasons it is best to pretend that it doesn't exist at all.
it doesn't seem too useful anyway (the user can simply mark the messages
as read to allow pruning).
and last but not least, the man page of mbsync says nothing about
"recent", only "unread". unlike the isync man page, though.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
6b7b2b1106 always get slave flags when we are expiring
even if we are not propagating new messages, the appearance of new
messages on the slave can lead to expiring older messages. for that, we
need to know their importance, and thus flags.

the alternative would be not doing an expiration run when not fetching
new messages, but that would mean more conditionals all over the place.
as the decision is somewhat arbitrary, just do the simpler thing.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
f1eea7d9a5 do not trash expired messages
we are not actually deleting them, so there is no point in saving them
in the trash.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
48754ecc74 make sync state header format less obscure
the header is not space-critical, so use proper name-value pairs.
this has the additional advantage that subsequent format changes can be
done much easier.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
83bb1cf716 make state loading in showstate() similar to ckstate() 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
f044adbfa4 take configs out of target state defs
cleaner and less duplication
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
5297425918 more precise failure reporting 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
3d81ccbf21 make it possible to run only selected tests 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
3dcb393de2 set srec->msg[] when finding messages by tuid
otherwise we would propagate phantom deletions.

this affected only sync runs after an interruption while storing
messages, so it went (mostly?) unnoticed.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
3814f19661 remove pointless assignment
we already know that tmsg->srec is null at this point.
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
e63e16ab45 assert no stray TUIDs 2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
8e49300cf7 initialize struct tm
strptime() does not initialize at least tm_isdst, which leads to an
uninited value reference in mktime().
2013-12-01 13:36:27 +01:00
Oswald Buddenhagen
62a60997c3 make use of strptime() portable
it does not (officially) support the %z conversion, so re-implement that
part by hand.
2013-12-01 13:36:26 +01:00
Oswald Buddenhagen
55e65147df fix compilation with older gcc versions
the warning suppression pragma within function scope is apparently a new
thing.
as i don't want to disable the check for the entire function (even if
this currently would make no difference), just use a wrapper function
to suppress the format string check.
2013-12-01 13:36:26 +01:00
Oswald Buddenhagen
32def5dc0a add/fix comments and improve debug messages 2013-12-01 13:36:26 +01:00
Oswald Buddenhagen
a9a331c98a simplify condition
... and document the cases.
2013-12-01 13:35:02 +01:00
Oswald Buddenhagen
03f8bfdfb2 micro-optimization/-clarification 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
00076a6971 move initializations for clarity 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
61ef099cd5 MaxMessages: make condition exactly symmetrical to condition below 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
080740f867 rewrite condition for readability and consistency 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
b10fd0c21c remove assumption about value of M constant 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
a893cba483 fix enum abuse
amends 9c86ec344.

S_FIND was for the sync record status field. it has no business in the
sync vars status fields. its value coincided with ST_SELECTED, which
luckily only means that we always tried to match up TUIDs even if there
was nothing to do.

the need for TUID matching arises in two mostly independent
circumstances, so add two separate flags ST_FIND_{OLD,NEW}.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
9a0e65f899 create unseen messages in /new/
seen messages still go to /cur/.
this is consistent with the actual maildir driver.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
0a684bd933 be a bit more verbose 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
394aca03a2 properly handle unexpected exit while replaying journal 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
07377cb753 ensure that the journal replay pass really does nothing 2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
100f9487f4 sort messages by serial number instead of UID in box dumper
the input data is sorted that way, so it's easier to compare.
2013-12-01 13:35:01 +01:00
Oswald Buddenhagen
2568459a7b fix error message in sync state dumper 2013-12-01 13:35:00 +01:00
Oswald Buddenhagen
b570c17766 deal with messages disappearing between being listed and fetched 2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
17c4748dfa support backslash-escaping in the config file
note that no attempt is made at making this work in the compat wrapper.
2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
2213d6976c support backslashes and quotes in quoted IMAP strings
the RFCs require it - well hidden in the BNF at the bottom.

patch somewhat inspired by "guns" <self@sungpae.com>.
2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
c0bf867669 make next_arg() more readable & efficient 2013-09-26 09:17:08 +02:00
Oswald Buddenhagen
3ceb553102 IPv6 support
inspired by a patch by "Todd T. Fries" <todd@fries.net>.
2013-09-01 17:39:07 +02:00
Oswald Buddenhagen
4a39cae8c4 support multi-homed servers 2013-09-01 17:39:07 +02:00
Oswald Buddenhagen
0b59ee0df3 support multi-character path separators
this applies to both the IMAP PathDelimiter (which is needed by Lotus
Domino), as well as the Flatten-ed separators.
2013-08-11 10:20:02 +02:00
Oswald Buddenhagen
1b67c49965 CHECK before FETCH after STORE
m$ exchange does not seem to update the index in time otherwise.
2013-08-11 10:20:02 +02:00
Oswald Buddenhagen
eb1f10762f added sync support for the arrival date of messages
initial patch by Marc Hoersken <info@marc-hoersken.de>
2013-08-03 18:54:34 +02:00
Oswald Buddenhagen
6577bf3e61 warn if we cannot find some messages by TUID 2013-07-27 20:18:20 +02:00
Oswald Buddenhagen
1847a4e12d make better use of ATTR_UNUSED 2013-07-27 18:44:26 +02:00
Oswald Buddenhagen
6dfccb76a5 be somewhat stricter about the LIST response syntax
the first token *must* be a list.
2013-07-27 18:40:19 +02:00
Oswald Buddenhagen
a0dc37339e allow the mailbox names in LIST responses to be literals
Lotus Domino seems to send them like that.
2013-07-27 18:40:16 +02:00
Oswald Buddenhagen
fc77feacc5 make parse_list() callback based
this allows us to parse IMAP literals ({}) in every list.
2013-07-27 18:39:39 +02:00
Oswald Buddenhagen
bf049d6466 add PassCmd option to query IMAP password dynamically
inspired by patches by
Aurélien Francillon <aurelien.francillon@eurecom.fr>,
Martin Stenberg <martin@gnutiken.se> and
sbfnk@users.sf.net.
2013-07-27 11:31:31 +02:00
Oswald Buddenhagen
5ad83b4e6a don't unnecessarily use continue 2013-07-27 09:34:17 +02:00
Oswald Buddenhagen
e4243debb6 use INT_MAX instead of zero for "no size limit"
this simplifies the actual conditions
2013-07-27 09:34:17 +02:00
Oswald Buddenhagen
ca3a319e60 update copyrights 2013-04-20 16:57:16 +02:00
Oswald Buddenhagen
01358ec8b4 man page fixups 2013-04-13 20:07:16 +02:00
Oswald Buddenhagen
406e967430 don't let wildcards match INBOX, unless it lives under Path
it's counter-intuitive to have '*' match the (always present) INBOX
when the rest of the mailboxes lives in a different namespace.
2013-04-13 19:58:50 +02:00
Oswald Buddenhagen
e7c96f8891 always list INBOX when asked for it
it's there even if it's not there. says IMAP. no need to contradict.
2013-04-13 19:57:41 +02:00
Oswald Buddenhagen
daaf950878 split maildir_list_part()
the boolean argument switched two entirely separate functions.
2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
167de3e438 remove some temporaries in maildir_list_part() 2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
842aa402c3 fix CRAM-MD5 authentication
the decoded challenge may be padded, so we really need to use strlen()
rather than just the decoded length.
2013-04-13 10:54:50 +02:00
Oswald Buddenhagen
e07de2a336 more consistency in char signedness 2013-04-13 10:54:50 +02:00
Felipe Contreras
aad7f903ec maildir: fix trash path double-free
It should be freed at the very end.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
2013-04-07 18:32:52 +02:00
Oswald Buddenhagen
89c81e382e discourage use of MapInbox 2013-04-07 16:57:45 +02:00
Oswald Buddenhagen
4ab55dc468 don't try to fetch status of minus one message
this would happen if we were trying to find newly pushed messages, but
none actually arrived.
as imap's ranges are not ordered, this would actually fetch one message.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
9261897629 don't record newuid in the sync state
this value is only ever used to find just pushed messages by TUID, so we
can simply use the UIDNEXT value from before we started pushing - and of
course, we need to record that in the journal. it makes no sense to log
the new value after completing a search, as there won't be a next search
before we push the next messages.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
96be183acb rename sync_vars_t::uidnext => newuid & fix comment
the purpose of this variable is to hold the UIDNEXT value from before
we started pushing new messages, i.e., the minimal uid we can expect
them to have.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
15d57b95b7 make paths relative to CWD, after all
the test suite actually relies on it. it would be possible to adjust it,
but there is not much reason to make paths relative to HOME (as we
support convenient tilde expansion). so use the least invasive approach,
which is simply the old behavior. adjust the documentation accordingly.

This reverts commit da5ce5d8f4.
2013-03-30 16:46:18 +01:00
Oswald Buddenhagen
7ba7be111e improve socket error reporting
always use getsockopt() to query the meaning of POLLERR, rather than
reporting "Unidentified socket error".
this is unlikely to have any effect when using select(), as that one
pretty much never signals exceptional conditions.
2013-03-29 18:25:39 +01:00
Oswald Buddenhagen
2ef6dc8a90 factor out socket_connect_failed() 2013-03-29 18:25:39 +01:00
Oswald Buddenhagen
37a28d8133 improve socket connect() error reporting with poll()
turns out that poll() may (and on linux does) signal POLLERR on
connection failure. this is unlike select(), which is specified to
signal write readiness in every case.
consequently, check whether we are connecting before checking for
POLLERR.
2013-03-29 18:24:32 +01:00
Oswald Buddenhagen
d1900941f4 introduce -DC option to only install a crash handler 2013-03-29 16:54:45 +01:00
Oswald Buddenhagen
b142778e56 match flag names in man page
"Full" is an alias for "All", but let's stick to one.
2013-03-24 18:42:57 +01:00
Oswald Buddenhagen
da5ce5d8f4 make path expansion match docu: paths are relative to ~
the current behavior of being relative to the current directory sort of
makes no sense, and contradicts the docu.
2013-03-24 18:42:57 +01:00
Oswald Buddenhagen
312f4be4b2 disable SSLv2 by default in the wrapper as well 2013-03-24 11:14:34 +01:00
Oswald Buddenhagen
89add4f330 downcast time_t to long for printing
time_t may be long long. to keep the sprintf format strings simple, just
downcast - this is not going to be a problem for the next 30 years, and
until then long will be 64-bit everywhere anyway.

suggested 3.5 years ago by Antoine Reilles <tonio@NetBSD.org>.
2013-03-24 11:14:27 +01:00
Oswald Buddenhagen
fbfcfea5dc fix cram-md5 authentication
we need to send a newline after the response for imap to grok it.
2013-03-23 10:38:27 +01:00
Oswald Buddenhagen
3363ad0f11 fix crashes in imap_open_store() error paths
it's not a good idea to invoke imap_open_store_bail() twice, either ...
2013-03-23 10:34:51 +01:00
Oswald Buddenhagen
a66034b23a fix crash in ssl connection error path
not a good idea to invoke the callback twice ...
2013-03-17 19:49:53 +01:00
Oswald Buddenhagen
acb1c870b4 rewrite SSL certificate verification. again.
leave all the hard work to OpenSSL. this has several consequences:
- certificate chain validation actually works instead of throwing
  around error 20
- the interactive approval is gone. i don't expect it to be useful
  anyway, as mbsync is mostly a batch tool
- the code is much shorter
2013-03-17 19:49:03 +01:00
Oswald Buddenhagen
8310cf78ac fix CVE-2013-0289: add SSL subject verification
we did not check a valid certificate's subject at all so far.
this is no problem if the certificate file contains only exactly the
wanted host's certificate - before revision 04fdf7d1 (dec 2000, < v0.4),
this was even enforced (more or less - if the peer cert had been
signed directly by a root cert, it would be accepted as well).
however, when the file contains root certificates (like the system-wide
certificate file typically does), any host with a valid certificate
could pretend to be the wanted host.
2013-02-17 18:33:04 +01:00