Home > Forum > Installation > Webcon DEV / TEST / PROD scenario

Webcon DEV / TEST / PROD scenario
0

Hello Webcon community!

We are wondering if there exist any common approaches / tools / patterns for transferring workflow data between different environments.

We have an on-premise environment for DEV (with internal AD) and an environment for PROD (which resides in Azure VM and connects to Azure AD).

What are your experiences to sync data, e.g. from PROD to DEV or TEST between different environments (different ADs) and is there even an easy
way of how to do it?


Thanks a lot in advance for sharing your experience,

Best regards, Nik

MVP

Hi Nik,

that's an interesting question. :)
Unfortunately, I don't have real experience with this in terms of WEBCON BPS. I've used this approach only with NAV and SharePoint. But these are quite similar in terms of transferring data from Prod to Test/Dev.

Let's break it down, we have:
- logic, part of the database
- data, part of the database
- privileges, part of the database
- website, can be ignored
- windows services, can be ignored
- search index, not sure whether this needs to be transferred in your case.

Logic/Data
Since logic and data reside in the same database and are inseparable the only option is to replace the whole database in Test/Dev or add a backup as an additional one. In my experience the replace approach is good enough for "updating" Test but not an option for Dev. You can freeze the deployments to Test so that there will be a point in time were the logic in Test and Prod matches so that you don't loose anything. This does not apply to Dev, in a lot of cases it will be contain updated logic which you don't want to lose. :) In most cases I needed new Data in Dev, when there was a bug in Prod or an performance issue. Adding an additional content database worked well enough in these cases.

Privileges
If there's a different source for users, the current privileges need to be replaced. For example you could fetch all users and replace specified logins using the ReplaceLogins stored procedure.
[dbo].[ReplaceLogins] @oldLogin = N'CCLS_RnD_Analyst', @newLogin = N'CCLS_RnD_Analyst@bps.local',
I don't know your data, so I can't tell whether this is a valid approach. Maybe your content database contains employee contracts which should be available in Test/Dev. Of course, you could delete those prior changing the privileges. :)

Environment
After transferring the database you will need to change the environment type:
https://community.webcon.com/forum/thread/366?messageid=366

Hm, I think I forgot a point but I just can't remember.:)

These are only my thoughts regarding this, and someone else can provide more input. :)

Best regards,
Daniel

Hello Nikolaus,


when you consider moving / copying data included inside processes I believe you may use REST Service to build your own application for inserting data.

Here https://github.com/WEBCON-BPS/RestApi-DataImporter - you may find sample application that allows you to export data from process to excel file on one environment (e.g. TEST ) and import the same excel on another environment (e.g. PROD).


Of course - we can not name this mechanism 'syncing' - it allows you to create new instances in efficient way - but it is still creation of new instances - so it means, that new instances will not have the same history, identical privileges, same instance number, same WFD_ID etc. Theese parts are strictly connected with specific of process so you need to take care of them on level of each process you are importing to.


If you consider syncing dictionary enrties (from dictionary process) please follow this thread
https://community.webcon.com/posts/post/importing-and-exporting-the-contents-of-dictionary-processes/251

__
Regards BS

In reply to: Daniel Krüger (Cosmo Consult)

Hi Nik,

that's an interesting question. :)
Unfortunately, I don't have real experience with this in terms of WEBCON BPS. I've used this approach only with NAV and SharePoint. But these are quite similar in terms of transferring data from Prod to Test/Dev.

Let's break it down, we have:
- logic, part of the database
- data, part of the database
- privileges, part of the database
- website, can be ignored
- windows services, can be ignored
- search index, not sure whether this needs to be transferred in your case.

Logic/Data
Since logic and data reside in the same database and are inseparable the only option is to replace the whole database in Test/Dev or add a backup as an additional one. In my experience the replace approach is good enough for "updating" Test but not an option for Dev. You can freeze the deployments to Test so that there will be a point in time were the logic in Test and Prod matches so that you don't loose anything. This does not apply to Dev, in a lot of cases it will be contain updated logic which you don't want to lose. :) In most cases I needed new Data in Dev, when there was a bug in Prod or an performance issue. Adding an additional content database worked well enough in these cases.

Privileges
If there's a different source for users, the current privileges need to be replaced. For example you could fetch all users and replace specified logins using the ReplaceLogins stored procedure.
[dbo].[ReplaceLogins] @oldLogin = N'CCLS_RnD_Analyst', @newLogin = N'CCLS_RnD_Analyst@bps.local',
I don't know your data, so I can't tell whether this is a valid approach. Maybe your content database contains employee contracts which should be available in Test/Dev. Of course, you could delete those prior changing the privileges. :)

Environment
After transferring the database you will need to change the environment type:
https://community.webcon.com/forum/thread/366?messageid=366

Hm, I think I forgot a point but I just can't remember.:)

These are only my thoughts regarding this, and someone else can provide more input. :)

Best regards,
Daniel

Hi Daniel!

Thanks a lot for your input, here are some interesting approaches for us.

We will need time to evaluate your suggestions.

[dbo].[ReplaceLogins] is also very interesting, does this function / procedure call affect / touch all items in the whole content database?

Best regards, Nik

MVP

Hi Nik,

the stored procedure ReplaceLogins replaces affects all tables where a user could be defined. Starting with business entities assignment over delegations, all fields in WFElements over securities. :)

I have on remark regarding the REST API Import approach. I assume that you not only need the started workflows but they must have been moved through to the respective steps.
If you do this, you must make sure, that the process version matches in both databases or there is no breaking change. For example:
- a path moves to a different step
- a path has been deleted
- a step has been deleted
- a field type has been changed

If you keep this in mind, use GUIDs to identify objects you could achieve to build a 'synchronization' application.

Best regards,
Daniel

In reply to: Daniel Krüger (Cosmo Consult)

Hi Nik,

the stored procedure ReplaceLogins replaces affects all tables where a user could be defined. Starting with business entities assignment over delegations, all fields in WFElements over securities. :)

I have on remark regarding the REST API Import approach. I assume that you not only need the started workflows but they must have been moved through to the respective steps.
If you do this, you must make sure, that the process version matches in both databases or there is no breaking change. For example:
- a path moves to a different step
- a path has been deleted
- a step has been deleted
- a field type has been changed

If you keep this in mind, use GUIDs to identify objects you could achieve to build a 'synchronization' application.

Best regards,
Daniel

Hi Daniel!

Thanks a lot for your time and your thoughts on this.

I think we will head for the content database method, since this is the only reliable method to ensure data consistency.

We do not need data on DEV environment, since this has already been prepared.

So in my opinion the best option to keep TEST/PROD in sync would be:

- Backup content DB and attachments DB in PROD
- Replacing content and attachments DB in TEST environment
- Creating local users on TEST server with the same signature (BPS_ID) and different (well known) password

I am not sure, if this is a good approach, maybe you already have some thoughts on that ;-)

Thanks a lot in advance & Best regards, Nik