DFSORT developer on retainer: CLMSORT SORT FIELDS wrong byte position after BATCH-TYPE field added shifts PAYER-ID from position 19 to 21 — downstream CLMPAY emits duplicate payer subtotals — IBM DFSORT developer, z/OS sort developer on monthly retainer

z/OS 2.4 • IBM DFSORT V1R15M0 • JCL CLMSORT • SEHE.CLAIM.DAILY • SORT FIELDS stale position • CLMPAY payer subtotals • retainer billing

At Southeastern Healthcare Exchange, the daily claims payment job had been running without incident for two years. Then a quarterly release added a 2-byte BATCH-TYPE field to the front of the claim record layout, shifting every subsequent field’s byte position by two. The COBOL copybook and all downstream programs were updated. The DFSORT SYSIN in the JCL job CLMSORT was not. CLMSORT still sorted by byte position 19 for the primary key, which in the new layout pointed into the middle of the newly shifted CLAIM-NUMBER field rather than at PAYER-ID. The sort output appeared normal — correct record count, correct record length, no DFSORT messages, RC=0 — and CLMSORT completed in the nightly batch window without any sign of failure.

The downstream COBOL program CLMPAY read the sorted file and used a payer-change detection loop to accumulate claim subtotals by payer and submit them to the clearinghouse. Because the sort sequence was wrong, PAYER-ID values were intermixed throughout the file rather than grouped. CLMPAY’s change-of-payer logic fired 847 times instead of the expected 23 times, producing 847 subtotal records — some with a single claim, some with a handful — many with the same PAYER-ID appearing in multiple non-contiguous groups. The clearinghouse received 847 separate remittance submissions; 23 payers received multiple partial submissions instead of one consolidated payment; 4 payers received zero submissions because their records happened to be scattered across groups that were attributed to adjacent PAYER-IDs in the garbled sort sequence.

DFSORT SORT FIELDS and the positional parameter model

IBM DFSORT is the primary sort/merge utility for z/OS batch. Unlike COBOL data division definitions or DB2 column names, DFSORT knows nothing about field names — it operates exclusively on byte positions and lengths within fixed-length or variable-length records. A SORT FIELDS specification identifies a sort key by three required parameters: the starting byte position within the record (1-based), the byte length of the field, and the format code that tells DFSORT how to interpret the bytes for comparison purposes:

  SORT FIELDS=(start,length,format,order[,start,length,format,order...])
  INCLUDE COND=(start,length,format,comparison-op,value[,connector,...])
  OUTREC FIELDS=(start,length[,format][,conversion...][,...])

Format codes for SORT FIELDS: CH for character (EBCDIC lexicographic), ZD for zoned decimal (COBOL DISPLAY numeric), PD for packed decimal (COBOL COMP-3), BI for binary (COBOL COMP or COMP-4), FI for signed binary, FL for floating point. Specifying the wrong format code causes DFSORT to interpret the field bytes using the wrong comparison semantics — a packed-decimal field sorted with CH will sort by its raw hex byte values rather than its numeric value. Specifying the wrong start position causes DFSORT to sort by whatever bytes happen to occupy that range in the actual record.

Neither error produces a DFSORT message or a non-zero return code. DFSORT sorts by what it is told to sort by. If the bytes at the specified position happen to form a valid comparison under the specified format, DFSORT proceeds. A wrong position on a CH-format key field is particularly silent: any byte sequence is valid as a character comparison key. The only way to detect the error is to compare the DFSORT SYSIN byte specifications against the current record layout and confirm that each specified position actually points to the intended field.

The SEHE.CLAIM.DAILY record layout change

The original 280-byte SEHE.CLAIM.DAILY record layout (before the quarterly release):

Byte  1 –  8 : PROCESS-DATE    ZD (YYYYMMDD)
Byte  9 – 18 : CLAIM-NUMBER    X(10)
Byte 19 – 28 : PAYER-ID        X(10)
Byte 29 – 58 : MEMBER-ID       X(30)
Byte 59 – 67 : BILLED-AMOUNT   PD (9 bytes, 17-digit packed)
Byte 68 – 76 : ALLOWED-AMOUNT  PD (9 bytes)
Byte 77 – 78 : SERVICE-CODE    X(2)
Byte 79 –280 : (remaining fields ...)

The quarterly release added a 2-byte BATCH-TYPE field immediately after PROCESS-DATE to distinguish EDI-origin claims (01), paper-origin claims (02), and resubmissions (03). The new 280-byte layout shifted every field after position 8 by two bytes:

Byte  1 –  8 : PROCESS-DATE    ZD (YYYYMMDD)
Byte  9 – 10 : BATCH-TYPE      X(2)   <<< NEW FIELD
Byte 11 – 20 : CLAIM-NUMBER    X(10)  (was 9–18)
Byte 21 – 30 : PAYER-ID        X(10)  (was 19–28)
Byte 31 – 60 : MEMBER-ID       X(30)  (was 29–58)
Byte 61 – 69 : BILLED-AMOUNT   PD (9 bytes)  (was 59–67)
Byte 70 – 78 : ALLOWED-AMOUNT  PD (9 bytes)  (was 68–76)
Byte 79 – 80 : SERVICE-CODE    X(2)   (was 77–78)
Byte 81 –280 : (remaining fields shifted by 2 ...)

The COBOL copybook SEHE.CLAIM.COPYBOOK was updated to reflect the new positions. Every COBOL program that accessed the claim file via the copybook picked up the new positions automatically at the next compile. CLMPAY was recompiled and tested. The JCL job CLMSORT, which embedded its field specifications as literal byte positions in a SYSIN DD card rather than via a copybook, was not identified as requiring an update. No mechanism linked CLMSORT’s SYSIN positions to the copybook’s field definitions.

The stale CLMSORT SYSIN and what DFSORT sorted instead

The CLMSORT JCL step, unchanged since the quarterly release:

//CLMSORT EXEC PGM=SORT
//SYSOUT  DD SYSOUT=*
//SORTIN  DD DSN=SEHE.CLAIM.DAILY,DISP=SHR
//SORTOUT DD DSN=SEHE.CLAIM.SORTED,DISP=(NEW,CATLG,DELETE),
//           DCB=(LRECL=280,BLKSIZE=5600,RECFM=FB)
//SYSIN   DD *
  SORT FIELDS=(19,10,CH,A,9,10,CH,A)
/*

In the new layout, bytes 19–28 no longer contain PAYER-ID. They now contain: bytes 19–20 = last two bytes of CLAIM-NUMBER (bytes 9–10 of the 10-byte field that starts at position 11), and bytes 21–28 = first eight bytes of PAYER-ID (which starts at position 21). The sort primary key is the last two characters of CLAIM-NUMBER concatenated with the first eight characters of PAYER-ID — a meaningless composite with no business significance. Bytes 9–18 in the new layout contain BATCH-TYPE (bytes 9–10) and the first eight bytes of CLAIM-NUMBER (bytes 11–18) — equally meaningless as a tiebreaker key.

The resulting sort sequence groups claims by an arbitrary composite that mixes two different fields. Because PAYER-ID values are spread across multiple claimed payers in non-alphabetic order with CLAIM-NUMBER digits interleaved, the sort output interleaves records from different payers throughout the file. DFSORT completed with RC=0 in 3 minutes and 47 seconds — a normal run time for the volume.

CLMPAY payer-change detection and the duplicate subtotals

CLMPAY processed the sorted file with a standard payer-change detection loop:

PROCEDURE DIVISION.
1000-MAIN.
    OPEN INPUT CLAIM-SORTED
    OPEN OUTPUT REMIT-OUTPUT
    READ CLAIM-SORTED INTO WS-CLAIM-REC
        AT END SET WS-EOF TO TRUE
    END-READ
    MOVE CLAIM-PAYER-ID TO WS-CURRENT-PAYER
    MOVE ZEROS TO WS-PAYER-TOTAL
    PERFORM 2000-PROCESS-CLAIMS
        UNTIL WS-EOF
    PERFORM 3000-WRITE-PAYER-SUBTOTAL    *> final payer
    CLOSE CLAIM-SORTED REMIT-OUTPUT
    STOP RUN.

2000-PROCESS-CLAIMS.
    IF CLAIM-PAYER-ID NOT = WS-CURRENT-PAYER
        PERFORM 3000-WRITE-PAYER-SUBTOTAL
        MOVE CLAIM-PAYER-ID TO WS-CURRENT-PAYER
        MOVE ZEROS TO WS-PAYER-TOTAL
    END-IF
    ADD CLAIM-BILLED-AMOUNT TO WS-PAYER-TOTAL
    READ CLAIM-SORTED INTO WS-CLAIM-REC
        AT END SET WS-EOF TO TRUE
    END-READ.

3000-WRITE-PAYER-SUBTOTAL writes a remittance record and submits it to the clearinghouse API. Because CLAIM-PAYER-ID values change every few records in the scrambled sort output, PERFORM 3000-WRITE-PAYER-SUBTOTAL is called 847 times instead of 23. Of those 847 calls: 341 produced single-claim submissions (PAYER-ID appeared alone before a different adjacent value), 384 produced 2–7 claim groups, and 122 produced groups of 8 or more claims. Across 23 actual distinct payers, some appeared in as many as 61 separate submission groups; others appeared in zero groups (their records were attributed to adjacent sort-key values).

The clearinghouse accepted each submission individually. Four payers received zero claims and no payment for the period. Nineteen payers received multiple partial submissions; three of those 19 triggered duplicate-detection rules at the clearinghouse and had their submissions held for manual review. The error was discovered 11 days later during a monthly reconciliation review when total submitted amounts did not match the daily claims totals.

Tracing the failure: comparing SYSIN positions to the copybook

The DFSORT developer retainer was engaged after the reconciliation discrepancy. Diagnosis: the retainer developer pulled the current SEHE.CLAIM.COPYBOOK member from the PDS and tabulated the byte offsets of every field. Then they compared each SORT FIELDS start position in CLMSORT SYSIN against those offsets:

SYSIN specification:  SORT FIELDS=(19,10,CH,A,9,10,CH,A)
                      Primary key:  position 19, length 10
                      Tiebreaker:   position 9,  length 10

Current copybook:
  CLAIM-NUMBER     PIC X(10)   position 11  (was 9  before layout change)
  PAYER-ID         PIC X(10)   position 21  (was 19 before layout change)

Primary key position 19 = bytes 9–10 of CLAIM-NUMBER + bytes 1–8 of PAYER-ID
Tiebreaker position  9  = BATCH-TYPE (2 bytes) + bytes 1–8 of CLAIM-NUMBER

Expected:  sort by PAYER-ID (pos 21) then CLAIM-NUMBER (pos 11)
SYSIN:     sort by (CLAIM-NUMBER tail + PAYER-ID head) then (BATCH-TYPE + CLAIM-NUMBER head)

The 2-byte shift was immediately apparent: every field after position 8 in the copybook was 2 bytes higher than the corresponding CLMSORT SYSIN position. The retainer developer also audited the INCLUDE COND and OUTREC FIELDS statements (there were none in CLMSORT, but other SORT steps in the job stream were checked). No other steps referenced the shifted fields.

The fix: updated SORT FIELDS positions

The corrected CLMSORT SYSIN:

//SYSIN   DD *
  SORT FIELDS=(21,10,CH,A,11,10,CH,A)
  * Primary:  PAYER-ID    at position 21, length 10
  * Tiebreak: CLAIM-NUMBER at position 11, length 10
/*

The comments were added as part of the fix — annotating each SORT FIELDS component with the field name and the copybook variable it corresponds to. A tested 100-record sample run against a copy of SEHE.CLAIM.DAILY confirmed that CLMPAY produced exactly 23 subtotal records, one per distinct PAYER-ID, with amounts matching the manual totals computed from the input file directly. The full daily file was reprocessed and the clearinghouse resubmission was coordinated with the operations team.

As a preventive measure, the CLMSORT SYSIN was augmented with a version comment that referenced the copybook layout date:

//SYSIN   DD *
  * Layout: SEHE.CLAIM.COPYBOOK version 2026-Q2 (LRECL=280)
  * PROCESS-DATE pos  1 len  8
  * BATCH-TYPE   pos  9 len  2
  * CLAIM-NUMBER pos 11 len 10
  * PAYER-ID     pos 21 len 10
  * MEMBER-ID    pos 31 len 30
  * BILLED-AMOUNT pos 61 len  9 format PD
  SORT FIELDS=(21,10,CH,A,11,10,CH,A)
/*

When the next layout change occurs, the version comment makes it immediately visible that the SYSIN positions need an audit. Without the comments, the only way to detect the mismatch is to independently read both the copybook and the SYSIN — which is what the DFSORT developer retainer did, at a cost of 2.5 hours of investigation time that the comment block would have reduced to 15 minutes of cross-checking.

How HourTab helps DFSORT developer retainers

The Southeastern Healthcare Exchange engagement split into two phases. The initial reconciliation investigation (11 days after the layout change): 1.0h clearinghouse submission analysis to count payer groups and identify the duplication pattern, 1.0h CLMSORT SYSIN vs. copybook comparison (including auditing all SORT steps in the job stream for other potential position mismatches), 0.5h fix (updating two start positions in SORT FIELDS plus adding comment block), 0.5h test (100-record sample, 23-payer confirmation), 0.5h clearinghouse resubmission coordination = 3.5h. Root cause documentation (separate session): 1.0h writing a DFSORT SYSIN audit checklist for all sort jobs in the daily batch stream = 1.0h.

In a HourTab retainer log, the investigation session reads: “CLMSORT SORT FIELDS stale positions after SEHE.CLAIM.DAILY layout change: BATCH-TYPE field added at position 9 shifted PAYER-ID from position 19 to 21 and CLAIM-NUMBER from position 9 to 11; CLMSORT SYSIN not updated; DFSORT sorted by bytes 19-28 (last 2 bytes of CLAIM-NUMBER + first 8 bytes of PAYER-ID) and bytes 9-18 (BATCH-TYPE + first 8 bytes of CLAIM-NUMBER); CLMPAY payer-change detection fired 847 times for 23 actual payers; fix: SORT FIELDS=(21,10,CH,A,11,10,CH,A); test: 23 subtotals confirmed; clearinghouse resubmission coordinated; SYSIN annotated with copybook version and field positions; 3.5h.” Southeastern Healthcare Exchange’s IT director can see exactly what caused the clearinghouse discrepancy and what was done to prevent recurrence.

HourTab turns a time-tracker CSV into a public retainer-hours URL your client can bookmark — no login, no portal, no “how many hours do I have left?” emails. Free for one retainer. $9/mo for up to ten.

FAQ: DFSORT developer retainers

See also: COBOL developer retainer · JCL developer retainer · VSAM developer retainer