SOLR indexing issues – TaskCanceledException & high CPU usage

0

Hi Martin,

Here are my recommendations. I faced a similar issue some time ago, so I hope this helps.
Solr usage shouldn't be directly tied to the SQL Server. All data sent to Solr is transferred via the Portal or Workflow Service.

You can check the status of Solr by visiting this URL:
http://webconserver:8983/solr/#/login
Make sure to replace "webconserver" with the actual server name.
The Solr credentials were set during installation (default username: Solr).

If your Solr collection is large and you're seeing "out-of-memory" errors in the logs, you may need to allocate more memory in the WebCon.BPSSearchService.exe.config file, typically located at:
C:\Program Files\WEBCON\WEBCON BPS Search Server

Update the memory setting like this - attached.

Regarding your points:
I recommend starting with a smaller scope - such as a single process. If that works successfully, you can then apply it to the entire database.
It's best to perform this task outside of working hours, as it may consume significant resources and affect search accuracy during active use.

For reference, most of this knowledge comes from support provided by @Wojciech Mleczko (WEBCON) - so credit goes to Wojtek!

Hi Adam,

Thanks for your reply. I’ll try updating the memory settings, but I suspect it won’t solve the issue. I’m not experiencing out-of-memory on SOLR level. I’ll proceed with reindexing, but first, I’ll test it in the test environment. I hope it solves the issue.

I’m open to other ideas if you or anyone else have any.

Cheers,
Martin

Hi Martin,

After upgrading in June this year WebCon BPS from Version 2017 (YES) to 2021.5.420, I had a similar situation (a large database of about 5 million records in WFElements and proportionally other tables) where SOLR was maxing out the processors and memory.
What helped:
- increasing the memory for SOLR to 8GB,
- removing everything it was struggling with, and reindexing process by process starting from the most critical ones (of course, reindexing on weekends and smaller processes after working hours).

All these actions helped, and today it runs stably, but the SOLR database is over 65GB, which seems quite a lot.

The next plan is to acquire a separate machine for a dedicated SOLR server to relieve the application server.
We still have the second part of the upgrade to the latest version ahead of us, but I hope that with a separate server, there will be no more problems or suprises with SOLR.

Most of this knowledge comes from WebCon support team.
Best regards.

Thank you both, Adam and Wiesław. I really appreciate your help. The memory increase resolved the issue. :)

Martin, if you have a lot of SOLR records and reindexing, you may consider separating the SOLR service onto a separate virtual machine so that it does not burden the WEBCON service.

Also, Changing the disk type from traditional HDD to NVMe improves Solr performance due to significantly lower I/O latency.