don't print the status, as the user doesn't really care whether it's
NO or BAD. more importantly, "NO LOGIN failed" is a rather misleading
thing to report.
replace the "DOM-like" model with a "streaming" model, i.e., the tokens
are now immediately processed by stateful callbacks as they are found.
this avoids plenty of allocations and copies (though not of the message
contents, yet).
- short literals are now printed even with only -Dn.
this ensures that we see all parts of, for example, LIST responses.
we use xprintf() for that, so we don't mess up the -Dn output in case
the literal contains something unexpected (we don't care with -DN).
- omitted bytes are now indicated.
- missing trailing newlines are now "pretty-printed".
- long literals are now printed in chunks as we receive them. this
makes the output messier, but it's necessary for an upcoming change,
and, on the upside, we'll get better indication when the transfer
gets stuck in the middle of a message.
this introduces a "commit" callback to the xprintf kernel, to avoid
pointlessly assembling a temporary output string prior to printing it.
one could lift the buffer limitations by introducing a "segment"
callback instead, but that would slow down xvasprintf() due to the
higher callback rate, for no good reason.
so far we shifted down the buffered data only when we ran out of space.
however, that may cause chopping up the actual socket read, which is a
bad trade-off with avoiding a moderate-sized memmove. so try to keep
enough space for the anticipated read size.
note that this ignores the zlib path, as that always does full-size
socket reads into z_buf, and saving inflate() calls seems pointless.
- use more appropriate name for socket object
- localize some variable declarations
- denoise the code by using more local variables
- don't pointlessly do stuff when we're failing anyway
technically, we should have been doing that since the beginning, but as
there is IMAP4rev2 now, it might actually matter (in about a decade,
when servers start dropping backwards compat ...).
so far, we'd print only a generic message - except in two cases, where
the generic error would be preceded by a specific one. now we always
print a single reasonably specific message.
the callbacks took the current parsing position only to pass it on to
the next list parse. so instead, store it in the state, and have a
separate function for continuation.
include the literal's length into the command itself, so error messages
become clearer. the commands are initially built for LITERAL+, and are
"edited down" right before sending if necessary.
some servers consider attempts to APPEND too big messages to be BAD
commands, rather than just NO-worthy ones. so just pretend that we did
in fact get a NO, and carry on.
this was tested with gmail. if we run into servers that don't send the
expected response code, we will need to relax the condition.
otherwise the synching may livelock due to failing to schedule follow-up
commands, thinking the buffers are still full. of course, this is
relevant only for commands that failed early and thus didn't free the
payload right after submission already - which will be possible only
after the next commit.
we didn't check that the UIDs are adjacent, so we might have caught
not fetched deleted messages between two fetched messages below the
bulk load range.
checking adjacency of UIDs would make expunges in the bulk range (which
is likely to be full of holes) rather inefficient. so we use sequence
numbers instead.
this is admittedly a rather academical fix ...
amends 18225344.
try to purge sync entries based on which messages are *actually*
expunged, rather than those that are *expected* to be expunged.
to save network bandwidth, the IMAP driver doesn't report all expunges,
so some entry purges would be delayed - potentially indefinitely, e.g.,
when only --pull-new --push is used, or Trash isn't used (nor
ExpungeSolo, prospectively). so keep a fallback path to avoid this.
this is essentially the same as 'New', but for previously seen messages,
such as those that would have been instantly expunged (because they were
marked as deleted), those that we failed to store for some reason, and
already expired ones that are now flagged.
REFMAIL: CAOgBZNonT0s0b_yPs2vx81Ru3cQp5M93xpZ3syWBW-2CNoX_ow@mail.gmail.com
change it from --{push,pull} to --{new,gone,flags,upgrade}.
that alone doesn't change anything; combining it with any other option
would cause a complaint anyway.
this enables us to introduce --{push,pull}-full, which will matter when
--full doesn't actually include all type flags any more, as we'll then
be able to combine it with the extra type flag(s). in the same vein, we
now advertize "Sync Full" instead of "Sync All" (both continue to be
recognized).
this is more symmetrical with New, and results in some less dodgy
grammar. it also avoids confusion with the \Deleted flag.
fwiw, the pedantically correct name would be Expunges, but that's
confusingly close to the target-side expunge options. also, it's longer.
when propagation of too big messages was entirely suppressed, the only
way to force it was flagging the source message. however, now that we
have placeholders that can be flagged to trigger full propagation, it's
rather pointless to keep the old method working, and still doing it
does in fact confuse users, see for example
REFMAIL: CAOgBZNq_a9yKcq8Jw5y9VS6p2Se8mD7gkf6vPr_KU0taAWuGZQ@mail.gmail.com
to avoid this, we now almost completely shadow the regular meaning of
flagging - it basically becomes a non-synchronizable flag until the
placeholder is upgraded.
this was broken by commit de6dc699 - we now iterated only over far-side
messages, which we don't necessarily load, unlike the near-side ones
(which we need to do to know their current importance).
fix by iterating over sync entries instead of messages, which basically
restores the pre-19128f15 state in that regard. the minor catch here is
that we now need an auxiliary array to sort the sync entries by UIDs. on
the upside, we can also use it to avoid repeated calculation of the
flags.
make sure that an expiration calculation run is performed if ongoing
expiration transactions have been loaded from the journal. this has the
nice side effect of centralizing the regular trigger condition as well.
flags_set_p2() is also adjusted to query S_NEXPIRE only if an expiration
is actually ongoing, like the flag propagation submission loop already
did.
this went unnoticed, because expiration upon arrival of new messages
wasn't autotested at all - despite being the common case.
when the expiration is interrupted, but an external expunge removes the
near-side message before we resume, we would just drop the transaction,
resulting in an "unmarked" orphan.
a corner case is an expiration that finishes, but initially isn't
expunged (probably due to an interruption), followed by an unexpiration
that gets interrupted, and the message being expunged externally
(because it's still marked as deleted). we obviously can't complete
that transaction without re-propagating the message, so effectively
cancel it instead.
