Page 1 of 1
REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Sun Sep 28, 2025 7:20 pm
by Anushka Sharma
Use case:
Reading ~1M lines and parsing.
Is EXECIO with STEM more memory-efficient than LINEIN?
Any sample loop patterns?
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Sun Sep 28, 2025 11:29 pm
by enrico-sorichetti
it depends on the parameters used
ADDRESS TSO "EXECIO * DISKR MYFILE (STEM MYRECORDS."
will read ALL the records , and a large file might hog the mvs memory an termiate the read
LINEIN will read one record at the time
P.S:
EXECIO will be faster when reading all the records in one invocation
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Fri Oct 10, 2025 8:09 pm
by Anushka Sharma
Makes sense, thanks! So in practice, would you suggest mixing both — say, using EXECIO in chunks (like 10K lines at a time) to balance speed and memory?
Or is LINEIN still the safer route for really large files?
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Fri Oct 10, 2025 8:23 pm
by enrico-sorichetti
I would use "EXECIO in chunks"
if You make the chunk size a variable to be set by an arg or statically at the beginning of the script
you will have a tunable environment by simply changing one statement
something like ...
Code: Select all
parse arg chunksize
if chunksize = "" then ,
chunksize = "a numeric value for the chunksize"
if DATATYPE(chunksize) /= "NUM" then do
say "ERROR, not numeric chunksize"
exit
end
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Thu Oct 16, 2025 11:58 am
by enrico-sorichetti
here is the proof of concept ...
written for open object rexx running on APPLE,
but it will run as i on any system supporting open object rexx
Code: Select all
#! /usr/bin/env rexx
Trace "O"
numeric digits 16
xiodata = "execio.data"
parse arg blksize
if blksize = "" then ,
blksize = 1
if DATATYPE(blksize) \= "NUM" then do
say "ERROR, not numeric blksize"
exit
end
-- outer loop
do blknmbr = 1
address hostemu "execio "blksize" diskr '"xiodata"' (stem xiobuff."
if ( RC \= 0 ) then do
say "EXECIO ERROR, RC="RC
address hostemu "execio 0 diskr '"xiodata"' (finis"
leave
end
if xiobuff.0 = 0 then do
address hostemu "execio 0 diskr '"xiodata"' (finis"
leave
end
-- inner loop
say
do recnmbr = 1 to xiobuff.0
say align(blknmbr) align(recnmbr) xiobuff.recnmbr
end
end
exit
align: procedure
arg n
return right(n, 3)
::requires "hostemu" LIBRARY
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Wed Jan 21, 2026 1:38 pm
by Anushka Sharma
Thanks, Enrico — super helpful.I tried the chunked approach locally and it looks promising. Before I finalize, wanted to validate a few things with the experts here:
- For ~1M lines, is EXECIO in chunks (say 10K–50K lines per read) generally the sweet spot? Or do you tune chunk size based on LRECL/avg line length?
- With STEM reads, is it okay to re‑use the same STEM per batch (overwrite stem. every time) or do you recommend dropping/clearing it explicitly after processing to avoid memory bloat?
- Any simple rule for when LINEIN is safer — e.g., when lines are very long/irregular or when address space is tight?
- On z/OS TSO: any knobs that actually move the needle — e.g., BUFNO, DYNALLOC/ALLOC FI() with BLKSIZE, or dataset attributes that help EXECIO throughput?
Below is the chunk pattern I’m testing (STEM overwrite per batch). If I’m missing something obvious, please point it out.
Code: Select all
/* REXX */
address TSO
parse arg dsname chunksize .
if dsname = "" then do
say "Usage: execio_chunk.rex '<dsname>' <chunksize>"
exit 8
end
if chunksize = "" | datatype(chunksize,'W') = 0 then chunksize = 20000
"ALLOC FI(INFILE) DA('"dsname"') SHR" /* or OLD if needed */
if RC <> 0 then do
say "ALLOC failed RC="RC
exit RC
end
tot = 0
do while 1 = 1
/* read a chunk into STEM */
"EXECIO "chunksize" DISKR INFILE (STEM REC."
if RC <> 0 then do
say "EXECIO read failed RC="RC
leave
end
if REC.0 = 0 then leave /* EOF */
/* process this batch */
do i = 1 to REC.0
line = REC.i
/* parse your fields here */
/* parse var line col1 1 col2 10 col3 +10 ... */
/* ...do work... */
end
tot = tot + REC.0
/* optional: free stem storage hint (overwrite next loop anyway) */
/* do i = 1 to REC.0; REC.i = ""; end; REC.0 = 0 */
end
"EXECIO 0 DISKR INFILE (FINIS"
"FREE FI(INFILE)"
say "Total lines processed:" tot
exit 0
For comparison, here’s the simple
LINEIN() loop I use when I want minimal footprint (yes, slower but steady):
Code: Select all
/* REXX (CMS/ooRexx compatible style) */
call linein 'MYFILE', 1, 1 /* position to start if needed */
tot = 0
do while lines('MYFILE') > 0
L = linein('MYFILE')
if L = "" & rc <> 0 then leave
/* process L */
tot = tot + 1
end
call lineout 'MYFILE' /* close */
say "Total lines processed:" tot
exit 0
``
- Use EXECIO in chunks as default for big reads, keep chunk size tunable via arg, and watch storage + elapsed.
- Keep STEM stable (overwrite each batch), avoid building giant global arrays.
- Fall back to LINEIN when memory headroom is tight or when per‑line streaming logic dominates.
Would love your thoughts on the above and any chunk size heuristics you’ve seen work well in production.
Thanks!
Re: REXX EXECIO vs LINEIN — when is EXECIO faster?
Posted: Tue Jan 27, 2026 12:59 pm
by enrico-sorichetti
topic locked, it is just spinning around with useless speculations