OrientDB throws exceptions randomly #10005
Comments
|
Here is another exception we receive - 2023-07-26 18:16:45:844 SEVER Exception |
|
Hi, I do have memory of similar error, but I cannot pinpoint if this has been yet fixed in a more recent version than 3.2.14, do you have an example query that cause the problem ? and as well it looks like it is an update, so do you have an example of the record before the update. Regards |
|
An important thing to know about this problem is that in 3.2.14 this bug would make the server read only stopping all the writes, but since the 3.2.16 (so also the latest 3.2.21) this error will make fail only the specific query and not stop all the other writes, which should improve a lot the situation even if this bug is still present in 3.2.21. Regards |
|
@tglman If the database goes into read-only mode then whether it executes the get/select queries. As, in our case if we get the above exception, then after that all queries are getting failed. Unfortunately, we observe this exception very randomly and still we are trying to reproduce this issue. I will update our findings here. |
|
Hi, I did check this case and I noticed that the ArrayIndexOutOfBoundsException exception could happen when reusing space that was too small to fit the data header, so I made sure to check the free space size and if the space is too small to be skipped, this should fix this problem and is going to be released next hotfix 3.2.22 Regards |
|
Thank you @tglman. |
|
Hi, The 3.2.22 has been just released, please upgrade to it and let me know if this resolve also your problem. Regards |
|
Thank you @tglman. |
|
@tglman / @praveen-mamdge is there any way to avoid this issue completely by doing some configurations like disabling cache and such? We are currently facing this issue with our 3.1.6 installation and we are not planning to upgrade at any point now. |
|
@gunaskr there was Mt issue. I have already fixed it |
|
You can't fix it without upgrade |
|
thanks for quick response @Laa |
|
@Laa we are getting this exception specifically on a certain vertex it seems |
|
As for your question @gunaskr , could you try to perform export/import. That will reorganize your data and may exclude problematic code paths from execution. |
OrientDB Version: 3.2.14
Java Version: Java 11
OS: Docker (Linux)
Description
Recently, we upgraded from 3.0.29 to 3.2.14 version. Randomly, we are getting following exception on the OrientDB console. It is not clear that which operation is causing this exception. Can someone please help on this?
2023-07-21 18:55:22:260 SEVER Exception
27CF2783in storageplocal:/orientdb/databases/ClientDB: 3.2.14 (build 947055a, branch UNKNOWN) [OEnterpriseLocalPaginatedStorage] com.orientechnologies.orient.core.exception.OStorageException: Internal error happened in storage ClientDB please restart the server or re-open the storage to undergo the restore process and fix the error. DB name="ClientDB" at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkErrorState(OAbstractPaginatedStorage.java:4588) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkOpennessAndMigration(OAbstractPaginatedStorage.java:4561) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.getPhysicalClusterNameById(OAbstractPaginatedStorage.java:3863) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.getClusterNameById(ODatabaseDocumentAbstract.java:632) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.addRecord(OTransactionOptimistic.java:476) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.saveRecord(OTransactionOptimistic.java:420) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.saveAll(ODatabaseDocumentEmbedded.java:1211) at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:208) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.saveInternal(ODatabaseDocumentAbstract.java:1486) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:1434) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:1317) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:80) at com.orientechnologies.orient.core.sql.executor.SaveElementStep$1.next(SaveElementStep.java:37) at com.orientechnologies.orient.core.sql.executor.CountStep.syncPull(CountStep.java:52) at com.orientechnologies.orient.core.sql.executor.OSelectExecutionPlan.fetchNext(OSelectExecutionPlan.java:35) at com.orientechnologies.orient.core.sql.executor.OUpdateExecutionPlan.executeInternal(OUpdateExecutionPlan.java:41) at com.orientechnologies.orient.core.sql.executor.ScriptLineStep.syncPull(ScriptLineStep.java:33) at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.doExecute(OScriptExecutionPlan.java:79) at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.fetchNext(OScriptExecutionPlan.java:40) at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.fetchNext(OLocalResultSet.java:36) at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.(OLocalResultSet.java:27) at com.orientechnologies.orient.core.command.OSqlScriptExecutor.executeInternal(OSqlScriptExecutor.java:129) at com.orientechnologies.orient.core.command.OSqlScriptExecutor.execute(OSqlScriptExecutor.java:68) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.execute(ODatabaseDocumentEmbedded.java:715) at com.orientechnologies.orient.server.OConnectionBinaryExecutor.executeQuery(OConnectionBinaryExecutor.java:1345) at com.orientechnologies.orient.client.remote.message.OQueryRequest.execute(OQueryRequest.java:143) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.sessionRequest(ONetworkProtocolBinary.java:355) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.execute(ONetworkProtocolBinary.java:239) at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:68) Caused by: com.orientechnologies.orient.core.exception.OStorageException: Exception during execution of component operation inside component clstercomponent_2.pcl in storage ClientDB DB name="ClientDB" at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:172) at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:157) at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.executeInsideComponentOperation(ODurableComponent.java:102) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.updateRecord(OPaginatedClusterV2.java:874) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateRecord(OAbstractPaginatedStorage.java:5071) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitEntry(OAbstractPaginatedStorage.java:5707) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2389) at com.orientechnologies.orient.core.storage.impl.local.paginated.OEnterpriseLocalPaginatedStorage.commit(OEnterpriseLocalPaginatedStorage.java:1198) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2214) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.internalCommit(ODatabaseDocumentEmbedded.java:1889) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:621) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:114) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:1597) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:1567) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.saveAll(ODatabaseDocumentEmbedded.java:1218) ... 22 more Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -2 is negative at java.base/java.lang.System.arraycopy(Native Method) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.serializeEntryChunk(OPaginatedClusterV2.java:503) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.serializeRecord(OPaginatedClusterV2.java:448) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.lambda$updateRecord$12(OPaginatedClusterV2.java:913) at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:165) ... 36 more 2023-07-21 18:55:22:260 SEVER Exception073A2A61in storageplocal:/orientdb/databases/ClientDB: 3.2.14 (build 947055a, branch UNKNOWN) [OEnterpriseLocalPaginatedStorage] com.orientechnologies.orient.core.exception.OStorageException: Exception during execution of component operation inside component clstercomponent_2.pcl in storage ClientDB DB name="ClientDB" at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:172) at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:157) at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.executeInsideComponentOperation(ODurableComponent.java:102) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.updateRecord(OPaginatedClusterV2.java:874) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateRecord(OAbstractPaginatedStorage.java:5071) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitEntry(OAbstractPaginatedStorage.java:5707) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2389) at com.orientechnologies.orient.core.storage.impl.local.paginated.OEnterpriseLocalPaginatedStorage.commit(OEnterpriseLocalPaginatedStorage.java:1198) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2214) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.internalCommit(ODatabaseDocumentEmbedded.java:1889) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:621) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:114) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:1597) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:1567) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.saveAll(ODatabaseDocumentEmbedded.java:1218) at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:208) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.saveInternal(ODatabaseDocumentAbstract.java:1486) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:1434) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:1317) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:80) at com.orientechnologies.orient.core.sql.executor.SaveElementStep$1.next(SaveElementStep.java:37) at com.orientechnologies.orient.core.sql.executor.CountStep.syncPull(CountStep.java:52) at com.orientechnologies.orient.core.sql.executor.OSelectExecutionPlan.fetchNext(OSelectExecutionPlan.java:35) at com.orientechnologies.orient.core.sql.executor.OUpdateExecutionPlan.executeInternal(OUpdateExecutionPlan.java:41) at com.orientechnologies.orient.core.sql.executor.ScriptLineStep.syncPull(ScriptLineStep.java:33) at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.doExecute(OScriptExecutionPlan.java:79) at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.fetchNext(OScriptExecutionPlan.java:40) at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.fetchNext(OLocalResultSet.java:36) at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.(OLocalResultSet.java:27) at com.orientechnologies.orient.core.command.OSqlScriptExecutor.executeInternal(OSqlScriptExecutor.java:129) at com.orientechnologies.orient.core.command.OSqlScriptExecutor.execute(OSqlScriptExecutor.java:68) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.execute(ODatabaseDocumentEmbedded.java:715) at com.orientechnologies.orient.server.OConnectionBinaryExecutor.executeQuery(OConnectionBinaryExecutor.java:1345) at com.orientechnologies.orient.client.remote.message.OQueryRequest.execute(OQueryRequest.java:143) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.sessionRequest(ONetworkProtocolBinary.java:355) at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.execute(ONetworkProtocolBinary.java:239) at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:68) Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -2 is negative at java.base/java.lang.System.arraycopy(Native Method) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.serializeEntryChunk(OPaginatedClusterV2.java:503) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.serializeRecord(OPaginatedClusterV2.java:448) at com.orientechnologies.orient.core.storage.cluster.v2.OPaginatedClusterV2.lambda$updateRecord$12(OPaginatedClusterV2.java:913) at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.executeInsideComponentOperation(OAtomicOperationsManager.java:165) ... 36 moreThe text was updated successfully, but these errors were encountered: