Automating data workflows with Analytics Canvas Online
Once your canvases have been finalized, you can automate a workflow to run on a schedule, at the click of a button, or through an API call.
Automations are created in Packages. Packages contain one or more Canvases. Running a package through any method, regardless of the number of Canvases it contains, counts as 1 automation run towards your quota.
1. To get started, click Automation in the left navigation menu, then select Canvas Packages. Click "New Package" to create your first automation.
2. The first step will be to add a name to this automation using the “Package Name” field. You can also add a description using the second field, if you want to.
Click “Add Cloud Canvas to your automation package” to select the Canvas or Canvases you want to run in this package.
3. A pop-up window that lists all the existing canvases will appear. If you have a lot of canvases you can use the search function to get there faster. Once you find the one that you’re looking for, click on the checkbox next to its name and then click ok.
4. You can see that the canvas was added to the list. Click on “Next Step:Define Schedule” to move forward.
Scheduled Automation Run
The most common option for automation is to run a package on a schedule. Packages can be run as frequently as every 10 minutes, but most common is a daily scheduled update.
⚠️ Each run consumes quota, so be mindful of high-frequency runs and be sure to exclude times when your team is offline, such as weekends, or after business hours.
5. In this section there are 2 variables you need to define: the days and the time when the automation will run. First you have to check the box next to “Scheduled automatic run.”
On the left, you have the days of the week, and you just have to check the box next to the days you want it to run. On the right, there is nothing right now; click on “Add time.”
6.There are two ways to add time. One option is to select “Specific time” and type the hour you want it to run and then click ok. This will add that particular time and you can click again on “Add time” and redo the process until you have added all of them.
The second way to do it is to select “Run repeatedly” and choose how often you want it to run. For example, if you want it every two hours, you can see on the right that the time was automatically added. When finished, click ok.
The scheduler can run as frequently as every 10 minutes for any given package.
7. To remove one or more selected times, check one or more times, then click "Remove Selected"
API Automation
This method of automation runs a package when the URL for the package is hit. It can be triggered by an API call from an application, from the click of a button in an Looker Studio report, or by simply pasting the URL into a web-browser and hitting enter.
API automation is ideal for reports that need to be updated outside of a regular schedule, or "On-Demand".
8. You will get back to this screen and now you’ll have the schedule created - the days and the hour when the automation will run during those days. Click on “Next Step:API Automation.”
9. In this step you can enable more advanced features. To enable an end-point, a URL that will run the automation package, check the box next to “Enable run by API.”
Copy the URL and use it to trigger the automation through an API call or through the click of a button in your reports.
The second option is “Web Hook on completion.” This will hit the specified URL based on the outcome of the package (success or failure).
You can skip this step altogether if you’re not using these; just click “Next Step: Notification.”
Notifications
By default, the user who creates the package will be notified in the event of a package failure. To add additional people to the notification list, or to be notified when jobs succeed, follow the steps below.
10. Here you can set up notifications. All failure will be sent to the package owner by default. If others should be notified, include them under Custom Email Notifications and select the notification types.
The last step is to click on “Next Step: Submit” and the automation will be created.
11. Now the automation package will appear on the page.
Before adding another package, it is advised to click “Run Now” to run the automation and verify that it will complete successfully. Upon completion, a log file will be generated for the package as a whole and for each individual canvas that ran.
You can edit the time, the Canvases in the package, the name of the package, and you can delete the package.
You can also run a job on-demand or stop a job that is running (abort).
You can review the API Automation links and the Notifications and Error Handling options by clicking on the relevant tabs.
After creating a package, it is a good practice to click the Run Now button and allow the package to run until completion. Review the automation log and the output to ensure the task completed as expected.
Syncing Canvases within the Package
When a Canvas has been put into the automation package for the first time, it will have the label "Up to date". This indicates that the Canvas in the Automation Package has not been altered or changed since it was added to the package.
Users can continue to work with the same Canvas in the Canvas editor, even though it is used in an Automation Package. The working copy, shown in the Canvas editor, is the "DRAFT" copy, while the copy in the Automation Package in the "Production" copy.
The changes made to the Canvas will not be reflected in an automation run until the package is synced. Until then, the label will turn yellow and say "Older Version"
To make the Draft copy the Production copy, click the "sync" icon. The label will then change to "Up to date".
After syncing a package, it is a good practice to click the Run Now button and allow the package to run until completion. Review the automation log and the output to ensure the task completed as expected.
Automation Package Logs
When an automation run occurs, either from an API call, a Scheduled run, or a Manual run (by clicking the Run button in the automation menu), a log file will be generated for the package as a whole and for each individual Canvas that ran.
The logs will detail the date and time that the package ran, the inputs that were loaded, and the exports that were generated. It will also contain details of any issues or errors that were encountered.
Logs are found within each Canvas package below the scheduler window.
An automation run may have one of the following status messages:
Complete: The package has run successfully and there are no errors.
Aborted or Skipped: Generally, if the package is skipping, you can press the stop button, wait for it to complete, then run it again. An aborted package is simply one where the user pressed the stop button to abort the currently running package.
Failed: One or more problems were experienced during the package run. All attempts to run the package have failed. Address the issues described by the logs before running this package again.
Support & Troubleshooting
Most often any issues with automation will be detailed in the log files.
If a package fails more than 10 times consecutively, the scheduler will be disabled for the package until the issue is addressed.
Troubleshooting Steps
If your automation package fails, take the following steps:
- Review the log file in detail. Often the last few lines, above the quota consumption, will indicate what failed.
- If it is safe to do so, manually run the task again by pressing the Run Now button. Often issues relate to uptime with one or more of the services involved in the task (Analytics Canvas, Google, Microsoft, Amazon, your database, etc.), and simply running the job again succeeds as the service issues have since been resolved.
- Review the detailed log file. If the issue is with credentials, update your credentials through Admin > Credentials
- If the issue relates to one or more blocks on the Canvas, or if you cannot identify the issue, open the underlying Canvas and run it manually (click the Run Now button). The offending block should turn red and provide an error message. After addressing the error, be sure the sync the package and run it again through the automation scheduler.
- If you are unable to resolve the issue, or the log results are unclear and retrying the job does not work, reach out to support.