Commit Graph

1231 Commits

Author SHA1 Message Date
Oswald Buddenhagen
6061de0ba6 autotest: use more legible format for test data
instead of specifying the mailboxes and sync state verbatim, use a
format which deals only with "subjects" (but no UIDs), and specifies the
whole state for each subject on a single line (exceptions prove the
rule).

the dumpers don't try to re-create the abstraction, as that's deemed
to be an unreasonable effort.

while rewriting most of the test data anyway, move it to the bottom of
the file, which is a more natural location for it.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
db66c4d746 autotest: remove show()
this somewhat crude way of generating test data will be obsolete soon.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
3040625a62 autotest: make state comparison more elaborate
don't abort the comparison if continuing makes sense, and try to be more
specific about the problems.

we give up if messages are excessive/missing or the subject is wrong,
as that touches upon the rather complex problem of diff optimization.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
7ce8c09145 autotest: support tests that don't modify the state file
a test run may (legitimately or not) modify only the mailboxes, or even
nothing at all. do something sensible in this case.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
96ee50d6ba autotest: transform ck*() => cmp*()
this avoids the ugly and error-prone repeated reading of the state
after a failure.

cmpbox() had to be made non-destructive on the box state.

readchan() had to be created.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
325551ce79 autotest: use ints where we mean ints
... instead of using strings. perl allows being sloppy, but it doesn't
really help understanding the code.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
fc0ad9eb65 autotest: use more appropriate data formats
parse the test data into hierarchical structures instead of using it in
its raw form. this is semantically cleaner and allows us to change the
input format more easily.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
697f35fd97 autotest: factor out readstate() from showstate() and ckstate() 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
e0c1a83fc1 autotest: make more use of readfile() 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
640b2a6649 autotest: factor out mkstate()
for clarity and structural consistency.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
9f9a2af959 autotest: let mkchan() take an atomic channel state
this is consistent with ckchan() and printchan().
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
7f38c5dc53 autotest: add support for strace-ing
works just like valgrind-ing.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
22a1df73e4 autotest: add support for starting at a specific test
... which is often useful for continuation after test data fixes.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
03a38e48d3 autotest: use sub-string matching for test selection
typing the full names is rather tedious when they get long.
also, using sub-strings, we can match multiple tests with one argument.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
27f0c47010 autotest: be more verbose about internal errors & warnings 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
929aa3281b autotest: don't die on failure to read state file
this can happen as part of a "regular" test failure.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
5d7f2c7461 autotest: remove excessive close()s from showstate()'s error path 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
254d2be9f4 remove pointless "lost track of ... messages" warnings
it doesn't really add any value when resuming after an interruption, as
incomplete propagations are perfectly expected, and are recovered from.
but it does make sense in the non-UIDPLUS path, where not finding just
stored messages indicates a problem.
so move the reporting from match_tuids() to msgs_found_new(), and remove
a redundant warning from the latter.

amends 6577bf3e.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
2b797fac61 delay TUID assignment less
we can't delay TUID assignment until after starting propagation if we
want to provide both safety and performance.

amends a0961d65.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
04e225c7ce make some maildir (error) messages more helpful
include the affected path.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
8e83649c33 slightly improve maildir rescan debugging 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
b9f0162642 make free_*_messages() loops less obfuscated
notably, free_maildir_messages() had a dead assignment.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
35375df63f don't put IMAP stores in SST_BAD state into the unowned list
nothing would ever recycle them, so they'd just waste space. so cancel
them right away.

amends 9d22641.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
ae3a61b668 clarify / micro-optimize cancel_sync() 2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
75113ef796 get rid of some redundant casts
amends c3d91ae1, 1b67c499, 9c86ec34, and 83ebe902+1039ee25.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
11352708b8 fix roff abuse in mdconvert man page
this one was missed in a33e4475.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
9356300952 convert licensing to SPDX
while at it, add/fix some licenses/copyrights/comments:
- it makes no sense to have a GPL exception in scripts
- ted did not contribute to the man page
- tst_timers is not part of the mbsync executable
- explicitly put the build system under GPL and add copyrights
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
72ba7ef125 bump copyrights
it's legally irrelevant, but whatever.
2022-06-19 16:01:29 +02:00
Oswald Buddenhagen
043a8b5835 documentation tweaks
manual:
- explain what "rename on move" really means
- reword "remote" to "opposite" to make it less confusing
  (possibly renaming TrashRemoteNew left as an exercise for later)
- mention example mbsyncrc
- consistently capitalize Store/Channel/Group where they refer to the
  respective configuration entities
- emphasize that SyncState may need a trailing slash (as we do for Path)
- fix missing mention of global usage/default for some options
example mbsyncrc:
- add big fat note that empty lines matter
- stop demoing deprecated options
- point out that CertificateFile is optional

REFMAIL: 877dd11jb3.fsf@angela.anarc.at
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
16db3498b3 fix .gitignores
- src/tmp/ is actually a symlink (and thus not a dir)
- autoconf now generates configure~
- the coverity result archive was missed so far
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
7a4a887b3c sort lists of trashed messages after journal replay
the messages are trashed in mailbox (and thus UID) order, and in
practice we expect the operations to complete in order. however, if
older messages need to be trashed after a journal replay, and we get
interrupted again, the next replay would produce an unsorted array,
and thus break the binary search.

amends 2bba9b9.
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
c1feba585a don't clobber message status when upgrading placeholders
we'd reset the possibly set M_RECENT flag, which would lead to
pointless maildir rescans.

amends 70bad661.
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
2e17f427a9 fix severing of msg -> srec links upon maildir rescan
amends 9c86ec34 (the relevant line was arguably misplaced there, and
should have been in 2277ecef or whereabouts already).
2022-06-19 16:01:22 +02:00
Oswald Buddenhagen
f74b4e0d11 fix --debug-crash suppressing the progress display
there isn't really a reason for that; DEBUG_CRASH is quite unlike the
other DEBUG_ flags.

note that the DEBUG_*_ALL flags are not checked, because they always
come with their corresponding less verbose flag anyway.
2022-06-19 16:01:19 +02:00
Oswald Buddenhagen
c9b52f5aec fix maildir driver's debug flag
copy-pasto; it certainly wasn't meant to respond to --debug-sync.
the problem was barely noticeable, as the maildir driver's only debugs
are in the rarely triggered rescan path, apart from the flags usually
being used en bloc anyway.

amends 0e1f8f9a.
2022-06-19 16:00:15 +02:00
Oswald Buddenhagen
9c2cd0abd8 plug memory leaks when OPEN_OLD_IDS is used with Maildir
amends 77acc268.
2022-06-19 16:00:15 +02:00
Oswald Buddenhagen
259132b7e7 plug memory leaks in imap_{store,trash}_msg() error paths 2022-06-19 16:00:15 +02:00
Oswald Buddenhagen
4c2fb74207 fix storing messages on non-UIDPLUS servers
the fetch mode needs to be set for messages.

amends 42f165ec.
2022-06-19 16:00:15 +02:00
Oswald Buddenhagen
ee9fd2f5c7 workaround iCloud IMAP bug
thanks to Sabahattin Gucukoglu <listsebby@me.com> for the thorough
investigation.

REFMAIL: 29C5E84D-5FE7-47BB-9A14-2EC34D3921C5@me.com
2022-06-19 16:00:09 +02:00
Oswald Buddenhagen
d6b9a139e4 re-issue IMAP CAPABILITY after authentication
... if the server didn't include a corresponding response code by
itself. required for the sorry excuse of an imap server that ms
exchange is.
2022-06-19 16:00:09 +02:00
Oswald Buddenhagen
b6c36624f0 work around "unexpected EOF" error messages at end of SSL connections
gmail apparently doesn't send a close notification (SSL_shutdown())
before closing the TCP socket.
2022-06-19 16:00:09 +02:00
Oswald Buddenhagen
6b22c837f6 overflow-check ranges[] in imap_set_range()
amends 3d64f167.
2022-06-19 16:00:09 +02:00
Oswald Buddenhagen
87c2ac1cc9 reserve enough UID ranges in imap_load_box()
in certain configurations, under very unlikely conditions (which are
practically impossible to control remotely), we'd overflow ranges[].
in a typical gcc build, the values (which are also practically
impossible to control remotely) would be written at the end of buf[],
which would be rather harmless, as only a tiny part of buf is used
subsequently. so i'm not classifying this as a security issue.

amends 77acc268.
2022-06-19 16:00:09 +02:00
Oswald Buddenhagen
bb5e98e9ec bump version 2021-12-03 11:56:16 +01:00
Oswald Buddenhagen
f2b1e80033 modernize configure.ac 2021-12-03 11:56:16 +01:00
Oswald Buddenhagen
e686f88318 don't complain about concurrent flagging as deleted
the result of propagating a deletion is flagging as deleted, so shut up
if the only remote change is exactly that.
2021-12-03 11:56:16 +01:00
Oswald Buddenhagen
51673214ab fix read beyond end of input in copy_msg_convert()
the input isn't necessarily null-terminated (it currently is for imap,
but not for maildir), so if the message ended somewhere within the
header field name, we'd read beyond its end, which theoretically could
cause a crash. no other adverse effects could result, as we'd stop
processing such a broken message right afterwards.

amends 70bad661.
2021-12-03 11:46:33 +01:00
Oswald Buddenhagen
127003ee37 reject unreasonably long mailbox names from IMAP LIST
this wasn't really a security problem, as the name mapping we actually
do does not change the string length, and the iteration was already
safe after the literal length fix, but it's still better to catch weird
input.
2021-12-01 10:07:40 +01:00
Oswald Buddenhagen
92921b1d3b reject messages that grow too large due to conversion
that shouldn't really be a problem, as we have 2GB of headroom, and most
growth would happen when sending an all-newlines message from maildir to
imap (due to CR additions), which is mostly non-critical. but better
safe than sorry.
2021-12-01 10:07:40 +01:00
Oswald Buddenhagen
bc15e571b6 report conversion errors directly in copy_msg_convert()
that makes it easier to report various conditions without introducing
separate error codes.
2021-12-01 10:07:40 +01:00