...
Attendees:
JV, Enrico, Sijie, Yiming
Minutes:
JV: we have some a leak on on the reader
from Slack:
we
ran
into
fullGC
and
zktimeouts
over
a
long
run
and
the
GC
shows
like
this359752
3:
1650087
79204176
java.util.HashMap
359753
4:
718190
74691760
org.apache.bookkeeper.client.PendingReadOp
359754
5:
823944
59323968
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
Pointing
to
PendingReadOp
protected
void
cancelSpeculativeTask(boolean
mayInterruptIfRunning)
{
if
(speculativeTask
!=
null)
{
speculativeTask.cancel(mayInterruptIfRunning);
speculativeTask
=
null;
}
}
But
we
never
assign
anything
to
speculativeTask
so
we
never
cancel
them
That
appears
to
be
one
issue
for
sure
do
you
guys
see
any
other
issues?
or
have
run
into
similar
problem?problem?
Yiming reports that at Twitter they saw something similar, but it was due to the bookie which was throttling the clients, which in turn they were queuing up the requests
Sijie: The problem should be around the fact we are not cancelling the PendingReadOp
JV has a quick fix
Sijie: about the 128bit ledger id. switching to 128bit should not have impact on performances. It will need some engineering work for the implementation.
JV: Sijie's proposal is fine, but how can we start wokring on this, it is a bug change
Sijie: We can split the work in smaller tasks, it is important to stay always backward compatible.
Enrico: we have to decide a name for 'scope' soon
JV and Sijie: oky for "scope", it sounds like a "workaround", but it is fine, let's start this thing soon.