Using GO SDK, by default a thread will flush all current stored impression in its cache each 30 seconds, however, there is a limit to the impression queue in SDK's cache, if the quq is full, event IMPRESSIONS_FULL is fired and the SDK will attempt to post the impressions to clear the cache. When the process try to flush all impression, the follow error is logged
Error flushing storage queue couldn't send message to task SubmitImpressions
The SDK is trying to send impressions at a higher rate than the posting thread is evicting them.
To resolve the issue, follow steps below:
- Increase the size of the impressions queue by updating the following parameter `Advanced.ImpressionsQueueSize`. default is 10k, maybe increasing it to 20k would improve things
- Increase the bulk size of the impressions post to split servers by updating the following parameter `Advamced.ImpressionsBulkSize`. Default is 5k. 10k would be a logical next step
- Decrease the period at which the sdk sends impressions to the split servers by adjusting the following parameter `TaskPeriods.ImpressionSync`. The default is 30 seconds which is definitely on the low high if you're sending a huge number of impressions. something along the lines of 5-10 seconds should help.
Note: Please keep in mind that these changes will slightly increase the memory usage of the SDK as well as the network traffic.