Pullup job sizing recommendations
To ensure optimal performance, efficient resource utilization, and successful job execution, you can fine-tune the job resource settings from the global job limits page. This is especially important when dealing with exceptionally large sets of data.
The following table shows recommendations on how to scale your job in different circumstances.
| Total rows | Total columns | Total cores | Total RAM |
|---|---|---|---|
| 1M | 50 | 4 | 6G |
| 10M | 50 | 6 | 16G |
| 100M | 50 | 16 | 160G |
| 250M | 50 | 40 | 400G |
Tip If your table is extremely large, use Parallel JDBC to improve the performance of the data loading stage when the Pullup job runs.
- Ensure that your Edge or Collibra Cloud site meets all system requirements.
- Create a Pullup job.