Hi, we've been having problems with SOLR lately.
Yesterday indexing got completely blocked and started throwing two messages in the service statuses:
1.
SolrNet
Category: Exception
Role: SolrIndexer
SolrNet.Exceptions.SolrConnectionException: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404 Not Found</h2>
<table>
<tr><th>URI:</th><td>/solr/BPS_Activities/update</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>Not Found</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>
</body>
</html>
at SolrNet.Impl.AutoSolrConnection.<PostStreamAsStreamAsync>d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.AutoSolrConnection.<PostAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.LowLevelSolrServer.<SendAndParseHeaderAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<<ProcessAsync>b__7_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptsAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<ProcessAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<>c__DisplayClass5_0.<<ExecuteProcessAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteWithLogAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteProcessAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ProcessStepsAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrIndexerInnerProcessor.<ProcessAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<>c__DisplayClass3_0.<<ProcessAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Activities.Activity.<RunInnerAsync>d__41.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Activities.Activity.<RunAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<ProcessAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<>c__DisplayClass2_0.<<RunAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Logging.SolrLoggingContext.<LogProcessAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Queues.Processors.ProcessorBase`1.<ContextActionAsync>d__40.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Queues.Processors.ProcessorBase`1.<ProcessingThreadActionAsync>d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Queues.Processors.ProcessorBase`1.<ThreadActionAsync>d__38.MoveNext()
2.
SOLR indexing failed
Category: Exception
Role: SolrIndexer
Whole process duration: 0,01s. Elements: 0, attachments: 0 (with content: 0, corrupted: 0).
Batch size: 500
Solr connection lost: 0
Exception at ID: 753178
SolrNet.Exceptions.SolrConnectionException: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404 Not Found</h2>
<table>
<tr><th>URI:</th><td>/solr/BPS_Activities/update</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>Not Found</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>
</body>
</html>
at SolrNet.Impl.AutoSolrConnection.<PostStreamAsStreamAsync>d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.AutoSolrConnection.<PostAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.LowLevelSolrServer.<SendAndParseHeaderAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<<ProcessAsync>b__7_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptsAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<ProcessAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<>c__DisplayClass5_0.<<ExecuteProcessAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteWithLogAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteProcessAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ProcessStepsAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrIndexerInnerProcessor.<ProcessAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<>c__DisplayClass3_0.<<ProcessAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Activities.Activity.<RunInnerAsync>d__41.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.Activities.Activity.<RunAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<ProcessAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.ActivityExecutors.QueueActivityExecutors.SolrIndexerActivityExecutor.<>c__DisplayClass2_0.<<RunAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Logging.SolrLoggingContext.<LogProcessAsync>d__8.MoveNext()
(31.03.2025 20:02:39) 'Deserialize activity model entity kept in the queue element' started.
(31.03.2025 20:02:39) 'Deserialize activity model entity kept in the queue element' completed in 0,00s.
(31.03.2025 20:02:39) 'Log currently processed SOLR activity indexation operation' started.
(31.03.2025 20:02:39) Indexing activity: saved instance ID: 753178.
(31.03.2025 20:02:39) 'Log currently processed SOLR activity indexation operation' completed in 0,00s.
(31.03.2025 20:02:39) 'Index BPS activity step' started.
(31.03.2025 20:02:39) 'Index BPS activity step' completed in 13,00s.
Exception at ID: 753178
Error occurred: SolrNet.Exceptions.SolrConnectionException: <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404 Not Found</h2>
<table>
<tr><th>URI:</th><td>/solr/BPS_Activities/update</td></tr>
<tr><th>STATUS:</th><td>404</td></tr>
<tr><th>MESSAGE:</th><td>Not Found</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>
</body>
</html>
at SolrNet.Impl.AutoSolrConnection.<PostStreamAsStreamAsync>d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.AutoSolrConnection.<PostAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at SolrNet.Impl.LowLevelSolrServer.<SendAndParseHeaderAsync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<<ProcessAsync>b__7_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at WebCon.WorkFlow.Service.SolrIndexer.Steps.SolrThreadAttempter.<MakeAttemptsAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Steps.IndexActivityStepWithoutCommit.<ProcessAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<>c__DisplayClass5_0.<<ExecuteProcessAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteWithLogAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at WebCon.WorkFlow.Service.SolrIndexer.Processors.SolrPipeline.<ExecuteProcessAsync>d__5.MoveNext()
at that time SOLR was working, I could enter Solr Admin-UI.
anyone had something similar?