Why Batch Jobs are Important to Your Next Capacity Planning Model
This week’s article on Capacity Planning looks at how to capture Batch Jobs requirements in your Capacity Plan. IT systems that use processor-intensive batch jobs on a daily/weekly basis need to factor in the resources that support these servers.
Business critical information and other data flows are dependent on these runs completing successfully. Failure to do so undermines the operation of the company and will compromise its ability to function.
Also, as Al Pacino says to Hilary Swank in Insomnia, “It’s your name that goes on the report.” Double-check everything.
Same applies to you – it’s your career, after all. Make sure you know how much power these batch runs need so you can plan accordingly. If you overlook this, and the batch job fails, you're the one who has to answer.
Where do Batch Jobs fit into Capacity Planning?
Most Capacity Plans need to factor in batch jobs.
These daily, nightly, weekly or monthly runs are performed on a schedule basis so information can be processed quickly and not interrupt or degrade system performance.
Most batch jobs are performed out of office hours as there is more processor power available. Most are executed according to a fixed schedule. If you need to change this schedule, let other depts know in advance. Don’t make decisions in isolation – ask around and
- Send out a reminder that you're going to change the schedule at least 2 days in advance.
- Send out another reminder the day of the schedule change and
- Send out another reminder 1 hour before the change
You never know who ignored your emails as is about to deploy a major upgrade to a customer.
What are the main types of batch jobs?
Batch jobs can be broadly grouped as:
- Daily Batch Runs
- Weekly Batch Runs
- Monthly Cut-off Batch Run
What is a Daily Batch Job?
These are executed daily, usually at a fixed time.
There may be some variations in what is executed each day depending on the day of the week (e.g. Friday) and the day of the month.
What is a Monthly Cut-Off Batch Run?
This special run is different from the Daily Batch Run.
Historical data suggests that these batch runs use more processing resources than Daily Batch Runs as they have more information to process and are usually (thought not always) mission critical.
What Constraints are there on Batch Jobs?
Batch runs must meet the batch window.
In other words, it must complete its tasks before the system goes back online.
When is the Batch window open?
The batch window is dependent on your operations requirements.
For example, you can measure processor utilization during the standard batch window of [10:00 PM] to [5:00 AM] to determine capacity.
What is Production Analysis?
Production analysis is performed by analyzing the Monthly Cut-off runs from the cut-off dates, for example:
To establish how the cut-off batch cycle has changed over this period, use the finishing time of a group of jobs, all of which must complete before the system re-start.
The elapsed time is dependent on the computer operator who may add/remove jobs depending on the batch job schedule, CPU availability and available time before online re-starts.
For capacity planning, you need to consider both CPU utilization and elapsed time.
The finishing times were as follows:
Date Completion Total Cases
1-Jan-2008 3:53 AM 1,340,536
1-Mar-2008 4:16 AM 1,495,576
1-Jun-2008 4:56 AM 1,533,037
1-Aug-2008 6:01 AM 1,766,254
How to improve the performance of your Batch Jobs?
One way to improve the performance of your batch runs is to setup a batch performance analysis workgroup.
Use this opportunity to create a candidate list of batch jobs that require performance optimization.
Consider the following attributes when examining the batch jobs.
- Are there parallel flows for batch execution?
- Does the batch job need to be completed before the system comes up?
- Examine batch performance service requests
- Execution times over the last 12 months
- Is this batch job on the critical path?
- Trend over the last 12 months
Use this input to create a candidate list of prioritized batch performance improvements.
Benefits of Prioritizing Batch Jobs?
The benefit of prioritizing/optimizing your batch runs is that it does improve the individual and total run time for Batch Jobs.
You can demonstrate how specific jobs were optimized and show the results are your status reports.
Monitor and update key batch jobs (and their priority) on a regular basis
Base this decision on upon performance metrics collected during batch operations.
Examine if you can separate different data processing requirements, so that data (e.g. print, email, web, finance etc) from the primary operational database.
Implementing this separation will improve performance on the database during batch processing and online processing.
These are a few pointers when writing the Batch Jobs requirement for your Capacity Plan.