How to safely reference a nullable activity output in Azure Synapse Pipelines and Azure Data Factory
Did you know that there's a null-safe operator in Azure Data Factory's expression syntax? Well, you'd be hard-pressed to find any documentation about it, so I wouldn't blame you if you didn't know.
Some time ago I asked this question on the MSDN forums: How to create condition on copy output property which may or may not exist. In this case I was trying to retrieve the filesRead output value from a copy activity. If the copy activity finds files to read and reads them, this output is returned. But if doesn't read any files for some reason (in my case, because the activity couldn't connect to the source), the filesRead property may not be written to the activity output.
If we were to reference the filesRead property in a subsequent activity (maybe set it to a variable), we'd use the syntax activity('CopyActivity').output.filesRead. But what if that property isn't there? Well, then you'll get an error when the expression is evaluated.
The expression 'activity('CopyActivity').output.filesRead'
cannot be evaluated because property 'filesRead' doesn't exist,
available properties are...
One solution here (which, interestingly, wasn't possible when I first wrote the above issue on MSDN) is to wrap the expression in a conditional statement to check whether the output does exist, and return a different output if not:
@if(contains(activity('CopyActivity').output, 'filesRead'),
activity('CopyActivity').output.filesRead, null)
However, there's something better we can do.
Introducing the null-safe operator
Fortunately, Martin Jaffer (a MSFT employee) responded to my plea on the MSDN forum, and enlightened me of the null-safe operator in ADF: ?.. That is, if you are dependent on an output property which may or may not exist, write your expression as:
activity('<myActivityName>').output?.<targetOutputProperty>
For example:
activity('CopyActivity').output?.filesRead
Much nicer!
See an application of this method in the video below.
