Graceful shutdown & resume for batch

[Aditi Patwal]

As part of my current work, I happened to run into a scenario where Need safe stop/resume for maintenance windows. How do you build checkpoints and restart? Appreciate any ideas or examples you can share.

[Anuj Dhawan]

Checkpoint/restart is a broad subject, and the implementation depends heavily on what the program is processing.At a high level, a checkpoint marks a known consistent point up to which processing has completed successfully. A restart restores the application to that point and continues without losing or duplicating work. The checkpoint therefore needs to represent both the application’s position and the state of every resource it updates.There are also different levels of restart:

1. Job-step restart - Restarting a failed or stopped JCL step using the scheduler or the JCL RESTART parameter. This is useful only when the complete step can safely be rerun.
2. Application checkpoint/restart - The program periodically saves its processing position and working state, then resumes from the last successful checkpoint.
3. Subsystem-managed recovery - A database or transaction manager participates in checkpointing, rollback and recovery.

The actual design will vary considerably:

• For Db2, checkpoints are normally aligned with COMMIT boundaries, with the restart key or processing position stored separately.
• For IMS DB batch or BMP programs, CHKP and XRST calls may be used, depending on the program type and processing design.
• For GSAM files under IMS, the GSAM checkpoint/restart facilities can help reposition the files during restart.
• For VSAM or ordinary sequential files, the application normally has to maintain its own restart information and handle file repositioning, duplicate output and partial updates.
• For MQ messages, external calls or multiple resource managers, additional coordination or idempotency may be required.

A safe planned stop usually means that the program receives or detects a stop request, finishes the current logical unit of work, takes a valid checkpoint, closes resources cleanly and exits with an agreed return code. Restart then begins from that committed checkpoint.

Could you clarify the environment involved - Db2, IMS DB, GSAM, VSAM/sequential files, MQ or a combination - and whether this is an IMS batch program, BMP or ordinary z/OS batch job? It would also help to know whether you need to restart the whole step or resume from within a long-running step.

[hollow_rowan69]

For maintenance windows, the safe pattern Anuj described is finish the current logical unit of work, take a checkpoint that covers both application position and every updated resource, then exit with an agreed return code. Db2 work should checkpoint on COMMIT boundaries with the restart key stored separately; VSAM or plain sequential files need the program to track its own position and handle duplicate output on restart. Job step RESTART alone is only safe when the whole step can be rerun without side effects. Agreeing the stop signal and return code with operations up front prevents a cancel that leaves half written files.