Buffer underflow exception during anti-caching test

[juyoung]

Hello,

Can I get your feedback on an issue related to anti-cache?
When I turn on anti-caching feature in H-Store, I sometimes run into an issue as below. The problem prevents multiple iteration tests from producing consistent results. Could you tell me if you have seen this kind of buffer underflow exception while testing TPC-C benchmark with anti-cache enabled? Since this happens only when I enable anti-caching feature, I guess that you may already experience and correct it. For running the benchmark, I set up a machine for H-Store server and another machine for client threads. When I establish both server and clients on a single machine, the problem does not occur.
Thanks for your any comment to address the issue in advance.

=================== error message =====================
[java] 12:30:44,430 INFO - -------------------------- BENCHMARK EXECUTE :: TPCC --------------------------
[java] 12:30:44,431 INFO - Starting TPCC execution with 12 clients [hosts=2, perhost=6, txnrate=100000]
[java] 12:30:51,448 INFO - Initialized 12 TPCC client threads in 7.02 sec
[java] 12:30:51,507 INFO - Letting system warm-up for 120.0 seconds
java: java.nio.BufferUnderflowException
java: 12:30:53,275 incoming-01 ERROR - VoltPort died due to an unexpected exception
java: at java.nio.Buffer.nextGetIndex(Buffer.java:498)
java: at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:355)
java: at org.voltdb.exceptions.EvictedTupleAccessException.(EvictedTupleAccessException.java:39)
java: at org.voltdb.exceptions.SerializableException$SerializableExceptions$5.deserializeException(SerializableException.java:79)
java: at org.voltdb.exceptions.SerializableException.deserializeFromBuffer(SerializableException.java:220)
java: at org.voltdb.ClientResponseImpl.readExternal(ClientResponseImpl.java:311)
java: at org.voltdb.messaging.FastDeserializer.readObject(FastDeserializer.java:137)
java: at org.voltdb.client.Distributer$NodeConnection.handleMessage(Distributer.java:273)
java: at org.voltdb.network.VoltPort.call(VoltPort.java:146)
java: at org.voltdb.network.VoltNetwork$3.run(VoltNetwork.java:545)
java: at org.voltdb.network.VoltNetwork$2.run(VoltNetwork.java:238)
java: at org.voltdb.network.VoltNetwork$1$1.run(VoltNetwork.java:202)

[linmagit]

Hi Juyoung,

Thanks for letting us know that! I've seen this problem. Now we're adding
new features (multi-tier anti-caching) to the anti-caching of H-Store. It's
almost finished but there are still some functions that are not completed.
We're fixing them now and you'll see a new patch for anti-caching soon.

For now, I would suggest turn-on the client.blocking feature of H-Store and
set client.blocking_concurrent to for example 200. Otherwise too high a
txnrate (100000 * 12 in your experiment) might cause this buffer overflow
when merging evicted tuples. If that does not work, please send me the
script you used to run your experiment. My email is lin.ma@cs.cmu.edu. Any
issues related to that are also welcome.

Lin

On Mon, Oct 5, 2015 at 5:48 PM, Juyoung notifications@github.com wrote:

Hello,

Can I get your feedback on an issue related to anti-cache?
When I turn on anti-caching feature in H-Store, I sometimes run into an
issue as below. The problem does not always show up but it is quite nasty,
preventing multiple iteration tests from producing consistent results.
Could you tell me if you have seen this kind of buffer underflow exception
while testing TPC-C benchmark with anti-cache enabled? Since this happens
only when I enable anti-caching feature, I guess that you may already
experience and correct it.
Thanks for your any comment in advance.

=================== error message =====================
[java] 12:30:44,430 INFO - -------------------------- BENCHMARK EXECUTE ::
TPCC --------------------------
[java] 12:30:44,431 INFO - Starting TPCC execution with 12 clients
[hosts=2, perhost=6, txnrate=100000]
[java] 12:30:51,448 INFO - Initialized 12 TPCC client threads in 7.02 sec
[java] 12:30:51,507 INFO - Letting system warm-up for 120.0 seconds
java http://client-localhost-01: java.nio.BufferUnderflowException
java http://client-localhost-01: 12:30:53,275 incoming-01 ERROR -
VoltPort died due to an unexpected exception
java http://client-localhost-01: at
java.nio.Buffer.nextGetIndex(Buffer.java:498)
java http://client-localhost-01: at
java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:355)
java http://client-localhost-01: at
org.voltdb.exceptions.EvictedTupleAccessException.(EvictedTupleAccessException.java:39)
java http://client-localhost-01: at
org.voltdb.exceptions.SerializableException$SerializableExceptions$5.deserializeException(SerializableException.java:79)
java http://client-localhost-01: at
org.voltdb.exceptions.SerializableException.deserializeFromBuffer(SerializableException.java:220)
java http://client-localhost-01: at
org.voltdb.ClientResponseImpl.readExternal(ClientResponseImpl.java:311)
java http://client-localhost-01: at
org.voltdb.messaging.FastDeserializer.readObject(FastDeserializer.java:137)
java http://client-localhost-01: at
org.voltdb.client.Distributer$NodeConnection.handleMessage(Distributer.java:273)
java http://client-localhost-01: at
org.voltdb.network.VoltPort.call(VoltPort.java:146)
java http://client-localhost-01: at
org.voltdb.network.VoltNetwork$3.run(VoltNetwork.java:545)
java http://client-localhost-01: at
org.voltdb.network.VoltNetwork$2.run(VoltNetwork.java:238)
java http://client-localhost-01: at
org.voltdb.network.VoltNetwork$1$1.run(VoltNetwork.java:202)

—
Reply to this email directly or view it on GitHub
#202.