I am calculating pearson's correlation coefficient in python script,in function continuously for no.of values in input arrays,and pushed/appended this correlation to another array. now i have to send/append/push/update this array of results into Grid variable for further calculations based on it but i am unable to update it to grid variable. how i can check its updated in grid variable? Also How can i use it @ transformation level?
5 Community Answers
Kalyan Arangam —
Unfortunately, I do not have much knowledge of a pearson-coefficient or its applications. I can probably help if you breakdown further explaining what you have in redshift and how you intend to apply the values held in your grid variable to data that you have in redshift. What is your end goal.
The grid variable is represented as a list-of-lists in python.
There are some examples of working with it
Acquiring a Grid variable – context.getGridVariable(‘mygridvar’)
Updating it – context.updateGridVariable(‘mygridvar’, [[‘list’,‘of’],[‘lists’,‘!’]])
Some documentation – https://redshiftsupport.matillion.com/customer/en/portal/articles/2917841-grid-variables?#header6
As for using grid-variables in a Transformation job – a grid-variable can be used provide metadata for multi-valued properties where we expect a list of columns. Like in a table input or any other component where you are expected to provide a list of columns. As far as i know, there is not way to apply values held in a grid-variable to any calculations.
One possibility is to iterate over the values held in a Grid variable using a ‘Grid Iterator’ component. Then call a transformation job for each of those values, passing a specific value into the transformation job.
Hope the above makes sense. As i mentioned earlier, please expand further on your use case on what is your end-goal and how you intend to apply the values in the grid variable to your data in redshift.
Also, we have a gmail address for you. Are you able to share the name of the company you work for? It would just help keep tickets from the same company together. Please Ignore this if you are personally paying for the instance.
I appreciate you may not want to share them over the public forum. Please email the details to firstname.lastname@example.org with subject “Ref#16093”. A new private ticket will be created which is visible only to you and support-team.