Appboard/2.4/builder/caching and polling - Revision history

imported>Jason.nicholls at 07:09, 14 December 2013

2013-12-14T07:09:40Z

← Older revision		Revision as of 07:09, 14 December 2013
Line 1:		Line 1:
	{{DISPLAYTITLE:Caching and Polling}}		{{DISPLAYTITLE:Caching and Polling}}
			[[Category:AppBoard 2.4]]
	[[image:AdapterFlow.png]]		[[image:AdapterFlow.png]]

imported>Jason.nicholls at 06:58, 1 October 2013

2013-10-01T06:58:36Z

← Older revision		Revision as of 06:58, 1 October 2013
Line 19:		Line 19:

	Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data. Caching is only of benefit if multiple clients are requesting the same data within the timeout window, or a single client happens to be polling more frequently than the timeout window.		Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data. Caching is only of benefit if multiple clients are requesting the same data within the timeout window, or a single client happens to be polling more frequently than the timeout window.


	== Configuration ==		== Configuration ==
Line 24:		Line 25:
	* Refer to the specific [[appboard/2.4/builder/data_sources\|Data Source]] pages for enabling '''caching'''.		* Refer to the specific [[appboard/2.4/builder/data_sources\|Data Source]] pages for enabling '''caching'''.
	* Refer to the [[appboard/2.4/builder/data_collections\|Data Collections]] page for enabling '''polling'''.		* Refer to the [[appboard/2.4/builder/data_collections\|Data Collections]] page for enabling '''polling'''.


	== Optimization ==		== Optimization ==

imported>Jason.nicholls: /* Optimization */

2013-10-01T06:58:14Z

Optimization

← Older revision		Revision as of 06:58, 1 October 2013
Line 34:		Line 34:
	# Consider how often the client will be polling for new data. For example, if the client is running a daily summary chart on a particular data source, simply updating the cache once per day may be sufficient for that purpose.		# Consider how often the client will be polling for new data. For example, if the client is running a daily summary chart on a particular data source, simply updating the cache once per day may be sufficient for that purpose.
	# Consider the complexity of the query that is being run against the server. If a simple query is being run, the cacheTimeout can be set to a low value (such as 1 minute or less) with minimal impact to the end-user.		# Consider the complexity of the query that is being run against the server. If a simple query is being run, the cacheTimeout can be set to a low value (such as 1 minute or less) with minimal impact to the end-user.


	=== Setting the Polling Interval ===		=== Setting the Polling Interval ===
Line 41:		Line 40:
	# Consider the Widgets that are using the Data Collection. If you have polling set to every 10 seconds for a Widget that is showing a monthly trend chart, this frequency of polling will most likely be causing unnecessary load on the client.		# Consider the Widgets that are using the Data Collection. If you have polling set to every 10 seconds for a Widget that is showing a monthly trend chart, this frequency of polling will most likely be causing unnecessary load on the client.
	# Consider how many records are being passed between the server and client at each polling interval. If the number of records being returned are on the order of tens and not thousands (numbers will vary for any installation, depending on client performance and network speed), then the client polling could be set to a very short interval. For example, returning a dozen records every 15 seconds should not impact performance. Returning hundreds of records every 15 seconds would slow the client.		# Consider how many records are being passed between the server and client at each polling interval. If the number of records being returned are on the order of tens and not thousands (numbers will vary for any installation, depending on client performance and network speed), then the client polling could be set to a very short interval. For example, returning a dozen records every 15 seconds should not impact performance. Returning hundreds of records every 15 seconds would slow the client.


	=== Queries and Testing ===		=== Queries and Testing ===

imported>Jason.nicholls: /* Definitions */

2013-10-01T06:55:54Z

Definitions

← Older revision		Revision as of 06:55, 1 October 2013
Line 18:		Line 18:
	In this sense '''polling''' specifically refers to Data Collections configured to have a polling interval. This is actually driven by the client only for visible widgets that have a polling Data Collection configured. This is the recommended way to have the Viewer client update the display without requiring the end-user reload the client.		In this sense '''polling''' specifically refers to Data Collections configured to have a polling interval. This is actually driven by the client only for visible widgets that have a polling Data Collection configured. This is the recommended way to have the Viewer client update the display without requiring the end-user reload the client.

	Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data.		Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data. Caching is only of benefit if multiple clients are requesting the same data within the timeout window, or a single client happens to be polling more frequently than the timeout window.

	== Configuration ==		== Configuration ==

imported>Jason.nicholls: /* Optimization */

2013-10-01T06:51:59Z

Optimization

← Older revision		Revision as of 06:51, 1 October 2013
Line 27:		Line 27:
	== Optimization ==		== Optimization ==

	=== Setting the ~~caching interval for a Data Source~~ ===		Clearly a balance has to be struck between keeping the client with up-to-date information and the total number of queries being performed by the AppBoard server to external data sources.

			=== Setting the Cache Timeout ===

	# Consider how frequently the data is being updated in the back-end data source. If the AppBoard server is querying source data that is being updated every 20 minutes, and the cacheTimeout is set to 300 (5 minutes), then this would not be optimally efficient since about 75% of the time the server would be re-fetching old source data that is already in the server cache. However, it should also be considered that if the source data is being updated every 20 minutes, and the cacheTimeout is set to 1200 (20 minutes), it is possible that the data in the cache could be up to 40 minutes old by the time it is cached again.		# Consider how frequently the data is being updated in the back-end data source. If the AppBoard server is querying source data that is being updated every 20 minutes, and the cacheTimeout is set to 300 (5 minutes), then this would not be optimally efficient since about 75% of the time the server would be re-fetching old source data that is already in the server cache. However, it should also be considered that if the source data is being updated every 20 minutes, and the cacheTimeout is set to 1200 (20 minutes), it is possible that the data in the cache could be up to 40 minutes old by the time it is cached again.
Line 34:		Line 36:


	=== Setting the ~~polling interval for a Data Collection~~ ===		=== Setting the Polling Interval ===

	# Consider how frequently the cache is being refreshed for the Data Source being used by that Data Collection. If the Data Source's cache is being updated every 5 minutes, and the Polling is set to 60 (1 minute), then this would not be optimally efficient since about 80% of the time the Data Collection would be polling the same old data that is already in the client's cache. However, it should also be considered that if the Data Source's cache is being updated every 5 minutes, and the Polling is set to 300 (5 minutes), it is possible that the data in the client could be up to 10 minutes old by the time it is refreshed. In this case, you might decide to set the Polling to 60 (1 minute), sacrificing efficiency in order to ensure that the data being displayed will never be more than 5 minutes old.		# Consider how frequently the cache is being refreshed for the Data Source being used by that Data Collection. If the Data Source's cache is being updated every 5 minutes, and the Polling is set to 60 (1 minute), then this would not be optimally efficient since about 80% of the time the Data Collection would be polling the same old data that is already in the client's cache. However, it should also be considered that if the Data Source's cache is being updated every 5 minutes, and the Polling is set to 300 (5 minutes), it is possible that the data in the client could be up to 10 minutes old by the time it is refreshed. In this case, you might decide to set the Polling to 60 (1 minute), sacrificing efficiency in order to ensure that the data being displayed will never be more than 5 minutes old.

imported>Jason.nicholls: /* Configuration */

2013-10-01T06:48:16Z

Configuration

← Older revision		Revision as of 06:48, 1 October 2013
Line 22:		Line 22:
	== Configuration ==		== Configuration ==

	~~Caching can be set for certain types of Data Sources~~. ~~On the "Connect" screen of the~~ Data Source ~~Wizard, the cache interval~~ for ~~that Data Source is controlled by the "cacheTimeout" setting. Enter the number of seconds to wait between each update of the server~~'~~s cache from the back-end data source~~.		* Refer to the specific [[appboard/2.4/builder/data_sources\|Data Source]] pages for enabling '''caching'''.
			* Refer to the [[appboard/2.4/builder/data_collections\|Data Collections]] page for enabling '''polling'''.

	Polling can be set for any Data Collection. On the "Configure" screen of the Data Collection Wizard, the polling interval for that Data Collection is controlled by the "Polling" setting. Check the "Polling" box to enable Polling. Enter the number of seconds to ~~wait between each update of the client's cache from the server. The default value is 60 seconds. The minimum polling value is 5 seconds, and~~ the ~~maximum value is 3600 seconds (one hour)~~. ~~To maximize performance,~~ Data Collections ~~are only polled when one or more Widgets that use that Data Collection are visible.~~

	~~To disable Polling~~ for ~~a Data Collection, uncheck the "Polling" box~~. ~~The data will be loaded once when the Widget is started, and never updated unless the browser is refreshed or there is an Action that updates the data in the Widget.~~


	== Optimization ==		== Optimization ==

imported>Jason.nicholls: /* Definitions */

2013-10-01T06:41:49Z

Definitions

← Older revision		Revision as of 06:41, 1 October 2013
Line 16:		Line 16:
	* an admin client (Builder) ''previews'' a data collection		* an admin client (Builder) ''previews'' a data collection

	In this sense '''polling''' specifically refers to Data Collections configured to have a polling interval. This is actually driven by the client only for visible widgets that have a polling Data Collection configured. This is the recommended way to have the Viewer client update the display without ~~having~~ the end-user reload the client.		In this sense '''polling''' specifically refers to Data Collections configured to have a polling interval. This is actually driven by the client only for visible widgets that have a polling Data Collection configured. This is the recommended way to have the Viewer client update the display without requiring the end-user reload the client.

	Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data.		Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data.

imported>Jason.nicholls: /* Definitions */

2013-10-01T06:41:00Z

Definitions

← Older revision		Revision as of 06:41, 1 October 2013
Line 12:		Line 12:

	* initially viewing a board with a client (Viewer or Builder).		* initially viewing a board with a client (Viewer or Builder).
	* one or more visible widgets configured with Data Collections set to '''poll'''. By default Data Collections are not configured for polling.		* one or more ''visible'' widgets configured with Data Collections set to '''poll'''. By default Data Collections are not configured for polling.
	* user interacts with the board resulting in actions (switching to a new board, server side filtering, etc...)		* user interacts with the board resulting in actions (switching to a new board, server side filtering, etc...)
	* an admin client (Builder) ''previews'' a data collection		* an admin client (Builder) ''previews'' a data collection

imported>Jason.nicholls: /* Definitions */

2013-10-01T06:40:17Z

Definitions

← Older revision		Revision as of 06:40, 1 October 2013
Line 12:		Line 12:

	* initially viewing a board with a client (Viewer or Builder).		* initially viewing a board with a client (Viewer or Builder).
	* one or more widgets configured with Data Collections set to '''poll'''. By default Data Collections are not configured for polling.		* one or more visible widgets configured with Data Collections set to '''poll'''. By default Data Collections are not configured for polling.
	* user interacts with the board resulting in actions (switching to a new board, server side filtering, etc...)		* user interacts with the board resulting in actions (switching to a new board, server side filtering, etc...)
	* an admin client (Builder) ''previews'' a data collection		* an admin client (Builder) ''previews'' a data collection

			In this sense '''polling''' specifically refers to Data Collections configured to have a polling interval. This is actually driven by the client only for visible widgets that have a polling Data Collection configured. This is the recommended way to have the Viewer client update the display without having the end-user reload the client.

			Regardless of the method of requesting data from the client, the server actually returns data based on the '''caching''' settings of the Data Sources. With caching disabled this means the AppBoard server will attempt to retrieve new data from the data source for every request. With a non-zero cache timeout the AppBoard server will deliver cached results if they exist and haven't timed out, otherwise it will fetch new data.


	~~<b>Caching</b> is~~ the ~~storing~~ of data ~~in memory on~~ the ~~AppBoard server. Caching rates can be different for each Data Source that is configured in AppBoard. Every time~~ the server ~~retrieves fresh~~ data ~~from a back-end data source, this places a load~~ on the ~~server~~. ~~By setting the~~ caching ~~interval,~~ the server will ~~store the most recent data and use that until the interval elapses, at which time it will get~~ new data.


	~~<b>Polling</b> is the updating of data in the client (e.g. browser) by retrieving~~ the ~~current~~ data ~~in the server's cache. Polling rates can be different~~ for ~~each Data Source configured in AppBoard~~. ~~Every time the client retrieves fresh data from~~ the server'~~s cache, this impacts the performance of the client. By setting the polling interval, the client will use the data that was retrieved from the last poll until the interval elapses~~, ~~at which time~~ it will ~~get~~ new data ~~from the server~~.


	== Configuration ==		== Configuration ==

imported>Jason.nicholls at 06:34, 1 October 2013

2013-10-01T06:34:04Z

← Older revision		Revision as of 06:34, 1 October 2013
Line 6:		Line 6:

	== Definitions ==		== Definitions ==

			AppBoard uses a model of demand driven data collection. This means that all requests for data originate from the client (Viewer or Builder) which the server then has to fulfil. In a trivial example, without any clients connected to the AppBoard server it does not perform any querying of data sources at all.

			In a real world example the typical events resulting in requests for data are:

			* initially viewing a board with a client (Viewer or Builder).
			* one or more widgets configured with Data Collections set to '''poll'''. By default Data Collections are not configured for polling.
			* user interacts with the board resulting in actions (switching to a new board, server side filtering, etc...)
			* an admin client (Builder) ''previews'' a data collection





	<b>Caching</b> is the storing of data in memory on the AppBoard server. Caching rates can be different for each Data Source that is configured in AppBoard. Every time the server retrieves fresh data from a back-end data source, this places a load on the server. By setting the caching interval, the server will store the most recent data and use that until the interval elapses, at which time it will get new data.		<b>Caching</b> is the storing of data in memory on the AppBoard server. Caching rates can be different for each Data Source that is configured in AppBoard. Every time the server retrieves fresh data from a back-end data source, this places a load on the server. By setting the caching interval, the server will store the most recent data and use that until the interval elapses, at which time it will get new data.