How does Quota work for Cloud Hosted Services?
Analytics Canvas Cloud Hosted Services consist of:
- Analytics Canvas Online
- Analytics Canvas Desktop/Cloud Automation
- Analytics Canvas Data Studio Partner Data Connector
Cloud Hosted Services operate on a quota system so that fees are closely tied to usage. The quota applies to the entire subscription and all users within it.
Quota usage is based on 4 categories:
- Automation Events: a workflow run on a schedule or an API call
- Platform Bytes Processed: use of our cloud to run automation jobs or ingest data
- Online Bytes Processed: any processing that involves BigQuery, including use of the Data Studio connector
- Storage: the sum of all data hosted within Analytics Canvas
4 Quota Measures in Analytics Canvas
Automation Events
Automation events are counted as triggered starts of one or more canvases in an automation package in Analytics Canvas Online and in Desktop/Cloud Automation. A trigger can be a scheduled start time or an on-demand run using the API Automation feature in Analytics Canvas Online.
Server Automation events do not count towards quota since they do not run within the Analytics Canvas cloud, however publishing an Analytics Canvas Smart table to Analytics Canvas Online from Desktop or Server will incur storage and platform bytes.
By default there are 150 automation events included in each subscription.
Platform Bytes
Platform Bytes are incurred when the Desktop/Cloud Automation service runs a Canvas, and when data is ingested into Analytics Canvas Online.
The amount of bytes processed can be measured by the bytes in the table. The total bytes of each table processed is counted towards the Platform Bytes in any ingestion or automation task.
There is a minimum of 5mb of platform bytes for each task. Some data ingestion routines use multiple tasks, such as loading data in partitions from GA4, or processing all block in a Canvas using Desktop/Cloud Automation.
Platform bytes are recorded in the logs for each Automation Event in Desktop/Cloud Automation, Server Automation, and Analytics Canvas Online Automation. Full detail Quota Use table found in Analytics Canvas Online under Admin > Subscription and Quota.
By default there are 5GB of Platform bytes included in each subscription.
Online Bytes
Online Bytes are incurred when the data is processed by BigQuery and are the sum of the bytes of the dataset that are processed. This is the same calculation as used by BigQuery.
In Data Studio, Online Bytes are consumed when the datasets are accessed by refreshing reports or requesting data from the Data Studio connector. This includes developing charts and tables and applying filters.
In Analytics Canvas Online, Online Bytes are incurrent when previewing, profiling, or processing data in a canvas. Online Bytes are NOT incurred when blocks are connected. They are only incurred when a user clicks a stub to generate a data preview, runs a data profile, or runs a full workflow that processes the data export.
By default there are 100GB of Online Bytes included in each subscription.
Storage
Data is stored for the Data Studio Partner Data Connector, and within Analytics Canvas Online both in Smart Tables and in cache for any active Canvases. Storage is simply the sum of all bytes stored under your subscription.
For Analytics Canvas Online, Smart Table versions can be managed within the each table in the Tables menu. Click Edit on the table, then click “Manage Previous Versions”.
Storage can be monitored under Admin > Subscription and Quota > Detailed Storage View.
By default there are 100GB of storage included in each subscription. To manage storage of each table, go to the Tables menu on the left menu.
You can sort tables by size to identify large tables. Then click the Pencil icon on a table to edit the table.
Next click to Manage Previous Versions to bring up the version control.
You can now adjust how many versions are kept, as well as select individual versions for deletion, or delete history based on a date range.
How To Monitor Your Quota
Quota is applied on a monthly basis based on the start date of your subscription. You can find the full details of your quota within Analytics Canvas Online under Admin > Subscription and Quota or within Analytics Canvas Desktop under Accounts > Subscription Admin (note you’ll need to be logged in to see this menu item). In both applications you will see your quota for the entire subscription, no matter where the quota was consumed.
Within the Admin section of Analytics Canvas Online, there is a section called Detailed Quota Use. The table and controls allow you to see how much quota was consumed for each Canvas, during each automation run, and for each Data Studio dataset.
On each Canvas in Analytics Canvas Online, there is an icon in the bottom right corner that will show you the quota consumed for that Canvas during the session.
Exceeding Quota
All admins on the account are alerted when any of the quota levels exceed a threshold of 80%, 90%, 95%, and when 100% of quota is reached.
When quota is met or exceeded, the services related to that quota are suspended. Your account will not be billed and you will not incur additional charges without your explicit knowledge and consent.
Quota is enforced against each metric individually. If you run out of automation package runs for the month but still have Online Bytes quota, Data Studio reports and data processing in Analytics Canvas Online will not be affected.
Included Quota + Quota Pricing
All Subscriptions include the Mega Cloud level of quota as shown on the pricing page. Trials have slightly more quota so that users can determine their needs prior to selecting a quota level.
Additional quota is available in two tiers, Giga Cloud and Tera Cloud. Subscriptions can have one or more of each quota tier to satisfy their data processing needs.
Quota is additive, so purchasing 1 x Giga Cloud quota would bring the total automation events up to 1,150 from the 150 included in the Mega Cloud plan.
Quota is available at two levels, Giga Cloud and Tera Cloud, and you can purchase multiple instances of each. For example, a customer may use 1 x Tera cloud + 1 x Giga cloud for $598/mo, while another customer may need just 2 x Giga Cloud for $198/mo.
Troubleshooting and Reducing Quota
To manage within your existing quota, consider the following:
- Run automation tasks on business days, during business hours to save on Automation events.
- Remove any blocks that don’t connect to an export when using Desktop/Cloud automation to save on Platform Bytes.
- Use the date filter controls on the main canvas in Analytics Canvas Online to limit the amount of Online Bytes processed. You can run the full date range when ready to export.
- While there may be many years of data available, often only the last few years are needed for analysis. Limit the history of the data ingested to save on Storage, Platform bytes, and Online bytes.
- Generate summary tables and perform incremental loads on them to save significant bytes processed vs running a full historic load each day. You can perform incremental loads on the input into Canvas Online, and with both Desktop and Online on exports to your databases and BigQuery.
- If you are consuming a large number of bytes in Looker Studio, review your reports to ensure that you have the right number of tables at the right level of granularity. For example, instead of having one large table process each element of the report, create smaller summary tables and use only the large tables when absolutely required (such as counting user_psedudo_id with GA4 event data).
If you have any questions or concerns about your quota, please contact support@analyticscanvas.com