Upsource last docker image - Error: An error occurred during flushing data to database

jetbrains/upsource:2018.2.1154

VM CentOS 7, 8GB RAM - Docker Version: 1.13.1

Hi all,

I hope I'm in the right place, otherwise please postpone.

For our projects I am currently testing Upsource (free). Installed is the last Docker Image. The config was set up with listen-port, base-url and heapsize (6GB). The project runs well for almost a week but then there are error messages like this one:

------------

[2019-02-04 07:20:29,197] ERROR r: update topology-1 essaging.impl.TopologyProvider - [127.0.0.1:10001]: failed to update messaging topology
[2019-02-04 07:20:29,202] [Upsource Frontend Error] java.lang.Error: An error occurred during flushing data to database
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DatabaseFlushExecutorImpl.consume(DatabaseFlushExecutorImpl.java:192)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.AbstractBufferingWriter.a(AbstractBufferingWriter.kt:135)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.AbstractBufferingWriter.flush(AbstractBufferingWriter.kt:107)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.CqlDatabase.flush(CqlDatabase.kt:245)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.CqlDatabase.flush(CqlDatabase.kt:224)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.CqlDatabase$myDatabaseFlusher$1.flush(CqlDatabase.kt:68)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DynamicBufferingTable.putValue(DynamicBufferingTable.kt:184)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.core.db.MessagingTopologyTable.updateNodeDescription(MessagingTopologyTable.java:44)
[2019-02-04 07:20:29,203] [Upsource Frontend Error] at com.jetbrains.upsource.messaging.impl.TopologyProvider$listenForChanges$1.run(TopologyProvider.kt:72)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2019-02-04 07:20:29,205] [Upsource Frontend Error] at com.jetbrains.upsource.util.NamedDaemonThreadFactory.a(NamedDaemonThreadFactory.java:34)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at java.lang.Thread.run(Thread.java:748)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] Caused by: java.util.concurrent.RejectedExecutionException: Writer to db 'UpsourceSystem' is corrupted due to unrecoverable problem.
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DatabaseFlushExecutorImpl.consume(DatabaseFlushExecutorImpl.java:170)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] ... 16 more
[2019-02-04 07:20:29,206] [Upsource Frontend Error] Caused by: com.jetbrains.upsource.db.api.query.DatabaseConnectionError: Database connection failure
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.ThroughDbMediator.submitWriteTask(ThroughDbMediator.kt:48)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.ThroughDbMediator.submitWriteFunction(ThroughDbMediator.kt:37)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.buffering.CqlWriteBuffer.flush(CqlWriteBuffer.kt:250)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.buffering.CqlBufferedQueue$StorageChunk$flush$1.doResume(CqlBufferedQueue.kt:47)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at kotlin.coroutines.experimental.jvm.internal.CoroutineImpl.resume(CoroutineImpl.kt:42)
[2019-02-04 07:20:29,206] [Upsource Frontend Error] at kotlinx.coroutines.experimental.DispatchedTask$DefaultImpls.run(Dispatched.kt:149)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.DispatchedContinuation.run(Dispatched.kt:13)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.EventLoopBase.processNextEvent(EventLoop.kt:140)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.BlockingCoroutine.joinBlocking(Builders.kt:70)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.BuildersKt__BuildersKt.runBlocking(Builders.kt:46)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.BuildersKt.runBlocking(Unknown Source)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at kotlinx.coroutines.experimental.BuildersKt.runBlocking$default(Unknown Source)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.buffering.CqlBufferedQueue$StorageChunk.flush(CqlBufferedQueue.kt:46)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.AbstractBufferingWriter.a(AbstractBufferingWriter.kt:183)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.AbstractBufferingWriter.access$flushChunk(AbstractBufferingWriter.kt:12)
[2019-02-04 07:20:29,207] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.AbstractBufferingWriter$tryFlush$$inlined$synchronized$lambda$1.run(AbstractBufferingWriter.kt:135)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DatabaseFlushExecutorImpl$Executor.a(DatabaseFlushExecutorImpl.java:249)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at __. db_UpsourceSystem .__.run(JavaGeneratorTemplate.java:43)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DatabaseFlushExecutorImpl$Executor.a(DatabaseFlushExecutorImpl.java:248)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.jetbrains.upsource.db.impl.DatabaseFlushExecutorImpl$Executor.run(DatabaseFlushExecutorImpl.java:231)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] ... 1 more
[2019-02-04 07:20:29,208] [Upsource Frontend Error] Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:10020 (com.datastax.driver.core.exceptions.ConnectionException: [/127.0.0.1:10020] Write attempt on defunct connection))
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:265)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:62)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:357)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler$SpeculativeExecution$1.onSuccess(RequestHandler.java:403)
[2019-02-04 07:20:29,208] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler$SpeculativeExecution$1.onSuccess(RequestHandler.java:384)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.Futures$6.run(Futures.java:1773)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:595)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.addListener(AbstractFuture.java:96)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1776)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.datastax.driver.core.GuavaCompatibility.addCallback(GuavaCompatibility.java:112)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.datastax.driver.core.GuavaCompatibility.addCallback(GuavaCompatibility.java:100)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler$SpeculativeExecution.query(RequestHandler.java:382)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:341)
[2019-02-04 07:20:29,209] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:138)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:120)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.SessionManager.execute(SessionManager.java:700)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:144)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.WriteSessionWrapper$executeAsync$$inlined$forEach$lambda$1.doResume(WriteSessionWrapper.kt:36)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.WriteSessionWrapper$executeAsync$$inlined$forEach$lambda$1.invoke(WriteSessionWrapper.kt)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.WriteSessionWrapper.executeQuery(WriteSessionWrapper.kt:44)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.datastax.driver.core.WriteSessionWrapper.executeAsync(WriteSessionWrapper.kt:36)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.buffering.CqlWriteBuffer$flush$2.doResume(CqlWriteBuffer.kt:250)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.buffering.CqlWriteBuffer$flush$2.invoke(CqlWriteBuffer.kt)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.ThroughDbMediator$submitWriteFunction$2.doResume(ThroughDbMediator.kt:37)
[2019-02-04 07:20:29,210] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.ThroughDbMediator$submitWriteFunction$2.invoke(ThroughDbMediator.kt)
[2019-02-04 07:20:29,211] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.WriteExecutor.execute(WriteExecutor.kt:14)
[2019-02-04 07:20:29,211] [Upsource Frontend Error] at com.jetbrains.upsource.database.datastax.ThroughDbMediator.submitWriteTask(ThroughDbMediator.kt:44)
[2019-02-04 07:20:29,211] [Upsource Frontend Error] ... 21 more

------------

The project has a size of more than 1GB.
Smaller projects have no problems.

There is no proxy in between, no firewall.

We also had the same problems on a natively installed Jetbrains upsource ( on Debian 9) as in the Docker.

There always seems to be problems with the database connection.

Can anyone help?

 

Regards, Tom.

 

Please sign in to leave a comment.