How to create WFA datasources with Python, c# or any other language

A colleague of mine had an issue with one of his datasources.  The issue wasn't the point, it was how he had created the datasource that was inspiring.  He actually created his CSV files outside of WFA and then in his datasource just copied them in.

$qtree_export_policy = "D:\WFA_Datasources\Qtree_Export_Policy.csv"
Get-WFALogger -message("Grabbing the file - " + $qtree_export_policy)
Copy-Item $qtree_export_policy .\

This made me think...

Self Healing Fabric with OCUM & WFA

Who doesn't love the concept of a self-healing system.  But what is self-healing ?  How can it be accomplished.  And can I make it fit my needs ?

In this "cookbook"-tutorial, you can find at least 1 method how it can be done.  But once you know the concept, the possibilities are nearly endless.

This idea is originally from Yaron Regev, so all creds go to him :).

Securely store credentials with powershell

I you have ever written PowerShell, you probably encountered the challenge that you need to store credentials for your script.  While debugging you probably hard code your credentials in your script, but once you have the script in production (manually or scheduled), you probably don't want them in the script in clear text.

This piece of code is pretty old but still does the trick.

How to create advanced workflows

Hi there,

so you like WFA and you want to learn more ?  Then chances are this tutorial is one for you.
This is also an non-branded recording of my Netapp Insight Session : 12721-3 - Learn How To Make Advanced Workflows at Insight Las Vegas - Berlin 2017.

This is the agenda :

1) Schemes and Dictionaries
2) Datasources
3) Debug WFA Powershell
4) Remote Powershell Sessions
5) Scripted Datasources
6) SQL Datasources
7) Custom Command + Reservations
8) RBAC & AD
9) Functions
10) Advanced Logging
11) Backup & document WFA

Tutorial : Use Wfa as a self-service portal

In my previous post, I have shown you my package on how to abstract user-input and how that allows you to create more user-friendly workflows.

In this youtube tutorial we are diving into that package and have a look what it does and what it can mean to you. download the sample excel from this demo

How to use WFA as a self service portal

If you ever thought about using WFA as a self-service portal, I'm sure you have bumped into a few challenges on how to make it user friendly.

If you want to design a good a userinput form, the absolute number 1 rule is "KISS" (keep it short and stupid).  So if you are going to present an operator or end-user with a short and simple input form, how can it kick off a long and complex workflow ?  If you want to omit all the technical information and decisions from the operator, how can WFA know what resources to pick ?

How to create workflow independent constants in WFA

If you're working with WFA for some time, you'll probably be using Constants.  And you should ! The problem is that constants are part of your workflow.  If you're a bit of a good developer, you should develop for re-usability and that also means that certain parameters should be externalized.  Settings like customer information (dns servers, domain names, etc...) are a perfect fit for an externalized constant. 

Let's have a look at our possibilities

WFA Docs generator 2.0

A new WFA Doc generator with most features you'll want :

• No need for installing Word
• Creates documents for :
• Workflows
• Commands
• Scheme's, dictionaries & datasources
• Exports Dar files (optionally)

WFA workflow - Install PSExcel and PSWord

When you want to work with Excel and Word documents from within WFA, you either choose to install Excel and Word, or, way better, you choose for the PowerShell Modules
- PSExcel
- PSWord

This workflow will install these for you on the WFA server on the right location.  The actual source files are safely hidden in the help content of the workflow.

Manage DFS from WFA (W2012+)

I already created a post for managing DFS, but that was for W2008 servers, where it still has to be done with dfsutil.exe.
Since 2012 there is better way.
DFS or Distributed File System, is often used in enterprise environments to virtualize your CIFS tree structure.  If you are automating storage, and if so, CIFS, you might also take it a little further and update your DFS.  This powershell code allows you to add/create, remove & offline/online your dfs links and target shares and get the targets of dfs link.

Debug WFA in your Powershell editor

I use this script all the time, when I'm debugging new wfa commands in powershell.

This script is run on the wfa server (my laptop in my case).  It loads the WFA libraries and adds the default functions
- Get-WfaLogger
- Get-WfaCredentials
- Connect-WfaCluster
- Connect-WfaController
- Set-WfaCommandProgress

WFA Command - advanced logging

I created a powershell module once (somewhere in this blog) that allows you to do some powerfull logging.  I always planned on showing you how it's done as a WFA command.

For my NetApp Insight sessions, I had to create such a command, and I'm already posting it here.

Logging on 3 levels :
- Database
- Logfile
- Eventviewer

Powershell Try Catch in nested code and how to

I was writing some powershell (quite a lot of powershell actually) today and had to go through the code of someone else.  It's always nice to see how other people are doing stuff.

So I saw this piece of code with try catch.  Although the piece of code worked, I didn't agree with the approach.

Let's see what the approaches are and what's the best way. Comments are highly appreciated.

WFA Volume AFF efficiency command

The WFA Volume commands allow you to set and modify most options, but apparently not AFF efficiency settings.   Hendrik Land and Hannes Bellmer created this command to fill this void.

Setting DFS Links in Powershell or Wfa (W2008)

DFS or Distributed File System, is often used in enterprise environments to virtualize your CIFS tree structure.  If you are automating storage, and if so, CIFS, you might also take it a little further and update your DFS.  This powershell code allows you to add/create, remove & offline/online your dfs links and target shares and get the targets of dfs link.   The code is using dfsutil, so it will still work on older 2008 servers.  Obviously you need to install the DFS role.   NOTE : There is another post on how to do this with remote powershell (Windows 2012+ only)
http://www.wfaguy.com/2017/08/manage-dfs-from-wfa-w2012.html 

How to query database using function in WFA

Sometimes you need to get information from the database (WFA cache) but you don't want it to happen in the userinput, and you don't want to bother using a custom dictionary.

Examples : get the next available lun id, import settings (instead of hard-coding customer information in your workflows).  It also adds a level of re-usability to your workflows.  And better, you can share settings between multiple workflows (dns servers for example, domain information, ...)

Credits : Tim Kleingeld

How to quickly test/debug MVEL code ?

If you're working in WFA, you'll most likely have used a function.  And maybe you wrote one as well (or tried at least).  Functions in WFA written in MVEL.  In fact, MVEL can almost be used every where in WFA.  And mvel is pretty neat as well.  It's expression language based on java.  And you can use java objects as well.

If you tried to create a function, you might have already bumped into some issues and had to jump back and forth from the code to the test button to figure out what is going wrong in your code.

Here's a neat trick I learned from Mr. David Korns.

MVEL and string splitting, what's up ?

I had to create an mvel function the other day that required some splitting and it was driving me nuts.  If I splitted with a "," (comma), it worked fine, but when it splitted with a "|" (pipe), it was funny business.  Other issues if you want to split on a "." (dot).

So what's going on ?

Wfa command to set SPN

This command allows you set a service principal name (SPN).  I needed this command a few years back to handle SVM-DR (Cdot 8.2).  In my case, cifs clients would work against a dns-alias, during the cutover, the dns-alias would flip from the old svm-name to the new svm-name.  Even if the DNS-flip is instantaneous to client (TTL 0, dns cache cleared, etc...), during a snapmirror-break-dr-activate from one CIFS-SVM to another CIFS-SVM, you have a kerberos issue.  Your cifs clients still have the old kerberos ticket and won't trust the new connection to the new SVM (different name).  That's why you need to unregister the old svm name and register the new instead.  (in a snapmirror relation, the svm-names must be different).

Custom SQL Server Function to Get By Delimiter (and index)

Had to look for over 30 minutes to figure this one out.  A simple split function in SQL does not exist. Almost all solutions point you to a table as result.  But I was looking for a simple function that would allow me to split a string by delimiter and let me pick a certain index.

Something like :

getByDelimiter('hi|ho|hey','|',1) => "hi"
getByDelimiter('hi|ho|hey','|',2) => "ho"
getByDelimiter('hi|ho|hey','|',3) => "hey"

I finally found a solution on the web (but had to bugfix it)

Backup and Restore WFA with Perl

My colleague (Warren Barnes) wrote this Perl script.  Apparently it is now possible to restore with rest as well.  (maybe it always was, but I couldn't figure it out).  So if I can find the time, I'll adapt my PowerShell version as well to restore with rest.

Convert an excelfile to json with PowerShell

In my blog I've been showing you how easily you can import excel files.
Of course, not everyone works with PowerShell.  Some might rather work with a language where excel is less easy or JSON would be a better fit.  And everyone probably has a windows machine running somewhere, right ? Well, with this script, you can convert your "simple" excel files to JSON. Once you have your JSON-file, the sky should be the limit. 

WFA Workflow to check iSCSI setup

For a customer I had to create this workflow to check/create the iSCSI setup between a host and a vserver.  This workflow does some excessive testing on both the host & vserver side and will re-mediate if needed.

Download the dar

WFA Command to check and remediate iscsi connections

During my workflow creation to check and re-mediate an iSCSI environment and had to create a final WFA command that would create/re-mediate the actual iSCSI connections.

WFA Command to install or uninstall a windows feature on a remote computer

In order to double check an iSCSI environment, one part was to make sure the Multipath-IO feature was installed.  With this command, you can install or uninstall a windows feature on a remote computer.



Download dar file

WFA Command to check vserver iscsi

For a customer I'm currently writing a pretty advanced workflow to double check the entire iscsi environment.  One part is check if the vserver is running ISCSI and if it has enough iSCSI ip addresses.  I'm also checking the iscsi ip's against a network address.

WFA Command to check ip addresses on a windows host

For a customer I'm currently writing a pretty advanced workflow to double check the entire iscsi environment.  One part is check if the windows host has enough iSCSI ip addresses.  In case of vmware that would be 1, in case of a physical machine that would be 2.

So parameters are :
- computername
- remote credentials
- network address
- network mask

WFA Command to install snapdrive (windows) on a remote computer

I had a request from a customer to prep & check the iscsi connections on remote windows hosts.  The first part is checking if snapdrive (SDW) is installed.  While I was at it, it thought I'd quickly write a command to remotely install snapdrive.  Boy was I wrong when I though quickly.  First there was the challenge of figuring out the snapdrive silent install.  When I figured out that part I bumped wall after wall to get this working.  But... I did get it working.

WFA command to set windows service (start, stop, restart)

To make sure iscsi is running fine on a windows machine, i had to check if the service was started.  I've made this command more general.  You can now start, stop or restart a windows service, whether localhost or remote computer.  Credentials are NOT the wfa credentials, but you can get them from the WFA credential store.

How to create a new KeePass entry with powershell and/or wfa

For a customer this week, I had to create a WFA command (PowerShell) that was able to generate a new password and then store it into KeePass.  This command is assuming that we use a "MasterKey" (=password).  You can also work with a key file, windows user and combination of those.  But this command is just with a MasterKey.  Adoption to work with the keyfile is minimal.

Parse a html page

A colleague of mine had to give a demo about logfile analyses and asked for my help in parsing an html page.  The html page shows data about the Belgian Air Polution (http://deus.irceline.be/~celinair/no2/no2.php?lan=nl#tabel).  He wanted a script that would create a log file of this data.  Nothing fancy, just some regex manipulations, some looping and your all set.

Change the version of a dar file

Ever got a dar file that had a higher version than you wanted ?

Now, if you are a clever guy, and you know the package might still work in the lower version, then you must have done the following :

Unzip the dar, Change the manifest file, Rezip the files, Rename to dar

I'm pretty sure, I know I did, you have made the common mistake of rezipping the top folder instead of files, breaking your dar,  Right :) ?

Anyhow, I bumped into the same issue over and over this week (my local wfa = 4.1 and my customer was on 4.0), so I (finally) build a powershell script (even better : a module) that will do this for you.

How to automate 7MTT with WFA

Did you ever use 7MTT ?  It's a great tool to transition from 7-mode NetApp to Cdot.  But if you need to insert a lot of volumes and projects, that's a lot of clicking.

Now, 7MTT has a CLI and that's a big start.  But it's not the best CLI and even so, if you need to add like a 1000 volumes, you want to keep your fingers crossed if you are planning to just copy-paste all those commands.

But, keep your hopes up.  End of last year I created a 7MTT Wfa package.

How to migrate 7-mode to C-dot with WFA (NFS & CIFS)

Ever wanted to migrate from 7-mode to Cdot with WFA ?
7MTT is great, but sometimes you might have a reason to migrate with WFA instead.

I've created a WFA package for this.

Qtree Cdot Commands with Unix Permissions & NFS Export Policy

The default Create Qtree command does not seem to contain the NFS export policy & Unix Permissions options.

In the dar, you have :
- create qtree extended command : allows you to optionally set the NFS export policy & Unix Permissions.
- set qtree export policy : does how it's named.

This is a 4.0 version dar.

Download the dar

Backup and restore a WFA server with powershell

If your WFA server is a key component in your environment, you will want to have backup.  Now WFA takes backups every day (keeping 5 versions), but I was always told : a backup is only a backup if it's in a different location.  So I created this PowerShell script a while ago to automate a backup/restore.  You call it a synchronization script if you'd like.

Advanced Input Checking in WFA with custom Errors

I'm at a customer and I was, once again, bumping into a very recurring issue : how to have advanced input checking.  In WFA you can have cascaded drop down boxes and some minor dependencies (using the input dependency), but every now and then it just doesn't cut it.

How to Build a NetApp ONTAP Lab… For Free!

Check out the free eBook "How to build a NetApp Ontap Lab... For Free!" from Neil Anderson

Neil is maintaining a Twitter Blog (@Flackboxtv) about "How to build your own cloud and/or start a data center career".

Have a look

Change service accounts on multiple servers with advanced logging

A colleague of mine had to update a service account of multiple servers. +100 servers. Thank god, he's a smart guy (all my colleagues are). So he said, I'm not going to do that manually, there must exist a PowerShell script to do that. And so he googled and found a script to do that, just a couple of lines actually. But it was for a customer and wanted a more polished version of it. Logging, error handling, the lots. So here is what I wrote for him.

Active Directory Datasource

AD Users & Groups datasource

Rewritten from scratch using powershell remote.
Which has the advantage that no plugins, features ore roles are required.  All code is executed on the adserver instead.

Custom WFA Command : Set volume autosize threshold percentage

Set volume autosize threshold percentage

When you modify a volume, these are 2 settings you can't modify in the default command.  So here is a custom command to do it.

Download DAR

WFA 4.1RC1 online

OnCommand Workflow Automation 4.1RC1 has been published to the NSS

See documentation for more information
External Software Download available at
Linux: http://mysupport.netapp.com/NOW/download/software/ocwfa_linux/4.1RC1/
Windows: http://mysupport.netapp.com/NOW/download/software/ocwfa/4.1RC1/

 

Workflow Automation 4.1 RC1 includes the following new features and enhancements:

• Support for JSON in REST APIs
• Support for collaborative development
• Added a new role as ‘Approver’
• Added Active Directory group support for category
• Support for Red Hat® Enterprise Linux 7.2
• Added REST APIS for listing workflow execution status
• Added REST APIs for schedule management
• Enhanced REST API documentation using Swagger
• Added a new user input type ‘Dictionary entry’
• Added inventory display for dictionary entries
• Support for custom REST end points