Monday, December 5, 2016

WFA and reservations, what is it and how does it work ?

If you know a little about WFA, you probably know that WFA gets all his information from external sources.  Most of the time, it gets its information from OnCommand Unified Manager (OCUM).  Now OCUM, re-scans its information every 15 minutes.    After that re-scan, you still need to wait for the WFA re-scan to happen.  Agreed, you can configure OCUM to update the WFA cache database right after its own refresh (reducing the LAG-time), but you will always have a potential 15 minutes LAG.  So how can WFA work around that problem ?  That's what you will learn in this article.
Think about this :
  • You create a new volume
  • A minute later, whether in the same workflow or in another workflow, you create a new lun on that new volume.
No way, OCUM can re-scan your storage, detect the new volume, update the WFA cache database in that fraction of time.  So how does WFA know that volume exists ?

See youtube movie below !

Reservations

That's where reservations come in.  If you edit a WFA-command, you will notice there is a reservation-tab and verification-tab.  It's in that tab the magic happens.


How does it work ?

In short : the SQL statement in the reservation tab gets evaluated when the command is ran successfully.  In case of a "create"-command, this will be an "insert" statement, a modify-command will be an "update" and a remove-command will be a "delete" statement.   Next time the volume-table gets queried, the insert-statement is executed on top of the result-set, thus including the latest created volume(s) (although not yet discovered by the datasource).  Same for updates and deletes.

In detail  (for the MySQL guru's) : When a command, like "create volume", is created with a reservation code, a stored procedure gets created on the MySQL Cache Database.   Each time the command is successfully executed, the variables (in green, in the picture above) are stored as a "reservation".

Each time the volume table gets queried, that stored procedure is used.  For each reservation, it will run that stored procedure, executing the INSERT, UPDATE & DELETE statements, thus modifying the result-set of the actual query.

Verifications

Now the verification tab contains a SQL Select statement, searching for the new object.  As soon as your datasource (OCUM, ...) picks up the actual change, the reservation is released (avoiding doubles).

Planning Phase

This method is not only helping you overcome the LAG between the datasource refresh and the WFA cache refresh, it does a lot more.

When a workflow is started, if first goes through the "planning phase".  In that planning phase ALL the steps of the workflow are processed, including ALL the reservations.  So WFA has already reserved all the creations, modification, deletions of every step of the workflow, BEFORE the workflow has even started.  This allows WFA to delay workflow execution and still guaranteeing that the resources you need are reserved when the workflows is actually executed.  This is handy for delayed workflows, but also for parallel execution with multiple operators.  Just assume you have 2 operators running the same workflow, trying to create the same volume and/or consuming the same available aggregate space.  Because the reservations are not run in parallel, only one of those operators will be able to run his workflow, the other will immediately hit the reservation and get the error that the object already exists (although it doesn't at that time).

Incremental naming 

The same technique is used when working with incremental naming.  WFA has the ability to create incrementing names (like volume01, volume02, volume03), each time the increment-code is ran, it will search the previous number and increase, all in the planning phase.  Only possible, thanks to reservations.

Not clear yet ?  Leave a message, I will be happy to help you out.

See the youtube tutorial below to understand better how it works.

1 comment :

  1. Hey WFAguy,

    Do you have any details on how and when reservations are cleared?

    Cheers!

    ReplyDelete