AndrewN0 Member since 2012 4 posts
Posted: 1 year ago
Last activity: 1 year 2 months ago

What data flow DSS are available to tune performance of batch execution?

I currently have a data flow properly configured to run in batch across multiple nodes.

Currently, in the topology (under decisioning -> infrastructure -> simulation landing page), I have enabled the thread count for the relevant nodes (3) and set the thread count to 3 (increasing it any more does not seem to yield that much more performance gains, possibly due to server hardware constraints/config? Please advise).

As it is right now, we are running about 50 records over the course of 1-2 minutes and of course want to speed this up and am wondering what is the limiting factor to the current performance.

I know previously I had to manually increase the batch size from 1 to 250 in 7.2.2 but we are currently in 7.3.1 so that may have already applied, if not - would be happy to try this.

Our system admins have noted during batch execution, the CPU utilization goes up to 91% as well.

Data Integration System Administration
Moderation Team has archived post
Share this page LinkedIn