Monday, March 16, 2020

Moving to the Cloud.

Tech
No, this isn't going to be all about sneezing while making cake icing - sorry! It'll be about a project I was on recently and why this sort of thing, while time-consuming, really is worthwhile.

The company I was working with had a rather old Pick system completely custom-written for all customer-facing functions, newer warehouse controls, and a variety of reporting systems. They already had an on-premises SQL Server instance to provide an acceptable interface for all the reporting software, and the Pick Basic programmers had created the output software to fill it with data.

Their problem had been that the Pick system was becoming more demanding of maintenance in its old age, so they had decided to upgrade their whole system by replacing the Pick system with a Microsoft Dynamics 365 system, which, of course, lives in the cloud.

Happily, one of the features of the Dynamics system is that pretty much any entity within it can be set to export values every so may seconds (or minutes, ...) to an external instance of Azure SQL Database (named by default "BYOD" !!). The database records (stored in tables defined in the export) include indications of source, date, and time of creation. This allows the developer of an SSIS package on-premises to pull the data, transform it, and use it to update tables in a local SQL Server instance.

The transformation processes can, of course, include not just aggregation by time or by customer, supplier, warehouse, article, etc., but also the change of units - singles to dozens or kg to lbs and oz ! For example, Dynamics might record that 12 items were picked from the warehouse whereas the reporting systems might work in units of a dozen.

So, to convert to using the Dynamics system one must first connect it to the various electronic reporting systems from warehouses, etc. But .... connect it in such a way that what it produces in terms of data resulting from any specific event must be identical to that produced by the legacy system. This sounds simple, but that's as far as the simplicity goes!

So in mid 2018 Microsoft announced an add-on to Dynamics 365 called "BYOD". This would be an Azure SQL Database instance, running beside the Dynamics instance. Entities withing Dynamics could be modified to maintain tables in the BYOD with their values, updating at regular time intervals.
The result of this is that an on-prem system can be fitted with some SQL Server jobs written using SSIS and these jobs used to pull data from the BYOD database. The data can be pulled down, loaded into local staging tables, and then transformed into the formats expected by the programs running off the on-prem system. Once you've got all that designed, implemented, tested, and working then you're ready to move over from your old system to the new cloud one.
The key word here is designed - that includes determining all the pieces of data that the down-stream programs (reporting, etc.) need and then finding them in the new cloud data system. Once you've done that then the job is relatively simple, although tedious and needing a lot of patience.

[UPDATE] March, 2020; It looks like I'm going to be doing another similar project soon ... more info to come. Just keep away from CoViD-19 !!

No comments:

Found Food

I have published quite a few recipes here on my blog over the last few years, and I hope that all my readers have tried at least some of the...