Refresh and scheduling

Every report shows a stored result. The refresh keeps that result current. It runs in a Map/Reduce script, never in a page request, so a slow query cannot time out a page or make a viewer wait for someone else's report.

#The schedule

The install adds a scheduled deployment, HSS - Advanced Report Refresh MR (Scheduled), that runs every 15 minutes. Each run picks up every report that is due. A report is due when:

• its refresh interval has passed since its last run;
• someone asked for a refresh; or
• a previous run was cut short, for example by an account restart, and has been stranded for more than an hour.

The author sets the Refresh interval on the Details step of the wizard. The choices come from Configuration. By default they are Every 15 minutes, Hourly, Daily and Manual only. A Manual only report runs only when someone asks for it, and when a never-run report is first opened.

Because the schedule runs every 15 minutes, a report refreshes within about 15 minutes of falling due. Choose the longest interval that still serves the report's readers. A daily summary does not need to run every 15 minutes, and every run uses the account's script processing.

#How a run works

1. Plan. The script counts the query's rows, writes a run plan on the report record, and splits the work into chunks of 25,000 rows.
2. Read and write. Each chunk reads only its own rows and writes its own result files. Chunks can run side by side, up to the deployment's concurrency limit.
3. Assemble. The last stage joins the chunks into one result. It lists every file in the report record's Result Files field, then writes the run's facts on the record: last run, row count, duration and status.

There is no row limit. The app splits a result into files of about 9 MB each in the File Cabinet folder report-cache, which is never available without a login. Each run writes files under its own name, so the previous result stays readable until the new one is complete. The run then removes the old files, and the next run removes any files left by a run that never finished.

A report page downloads the result one file at a time and shows its progress as it goes.

#One run per report at a time

A scheduled run and a Refresh click can reach the same report at the same moment. To prevent a clash, a run claims the report before it starts. A second run that finds the report claimed leaves it alone. A run only ever updates, fails or cleans up a report that it claimed itself. For a viewer, this means that clicking Refresh while a scheduled run is already working has no visible effect. The report updates when that run finishes.

#Refreshing on demand

Each of these marks its reports as wanting a run and then starts one refresh job for all of them. If that job is already busy, nothing is lost. The reports stay marked, and the scheduled run builds them within fifteen minutes.

A short cooldown stops the same report being refreshed over and over. By default it is 120 seconds, set by refresh.manual_cooldown_seconds. A click inside the cooldown shows a "refreshed recently" message, which is not an error.

#Full and partial refresh

Full re-runs the whole query and replaces the result. It is the default, and it is right for most reports.

Partial fetches only the rows added or changed since the last run and merges them into the stored result, one file at a time. Set it on the Details step with a Unique identifier column and a Last modified column. Each run re-reads from a little before the stored watermark, 60 minutes by default (refresh.incremental_overlap_minutes), so the next run still catches a change made during a run. The run merges re-read rows by key, so the overlap costs query time, never accuracy.

Partial refresh never removes a row that was deleted in NetSuite. Full Refresh does. An imported report always starts with a full run, because the app stores the watermark with the result, not with the definition.

#What the record tells you

The report record keeps its run history where people can see it: status, last run, row count, the duration of its own read and write, and the last error. A report that has been failing shows it on its card in the catalogue. For a step-by-step trace, open the execution log of the refresh script. Every step writes a line titled AR refresh <id>: <step>, with row counts and timings.

#If something looks wrong

Deleting a report removes its result files too.

Related: Writing SuiteQL that performs · The report wizard step by step · Settings reference · Reading a report