Question

2
Replies
273
Views
Close popover
Andrew N (AndrewN0)
N/A

N/A
CA
AndrewN0 Member since 2012 4 posts
N/A
Posted: July 11, 2019
Last activity: July 17, 2019
Closed

What data flow DSS are available to tune performance of batch execution?

I currently have a data flow properly configured to run in batch across multiple nodes.

Currently, in the topology (under decisioning -> infrastructure -> simulation landing page), I have enabled the thread count for the relevant nodes (3) and set the thread count to 3 (increasing it any more does not seem to yield that much more performance gains, possibly due to server hardware constraints/config? Please advise).

As it is right now, we are running about 50 records over the course of 1-2 minutes and of course want to speed this up and am wondering what is the limiting factor to the current performance.

I know previously I had to manually increase the batch size from 1 to 250 in 7.2.2 but we are currently in 7.3.1 so that may have already applied, if not - would be happy to try this.

Our system admins have noted during batch execution, the CPU utilization goes up to 91% as well.

Data Integration System Administration
Moderation Team has archived post