Find the number of records in a Tape File without reading it.
-
- Registered Member
- Posts: 11
- Joined: Thu Jan 28, 2016 9:03 am
Find the number of records in a Tape File without reading it.
Hi,
We have couple of tape files, some of which are archieved and some of them are not. For an audit requirement, we need to see the number of records on these files. Can you please suggest a simple method of finding the number of records. We have two programs written in COBOL which can read the files and can get the number of records but with these tape files are having records around 16-18 million records each and cobol program as such takes long time to read it and generate a report on counts.
We also though of using SORT but with sortout as dummy it does not really help with the execution time. Can there be a way to count just records with less time used.
Thanks
We have couple of tape files, some of which are archieved and some of them are not. For an audit requirement, we need to see the number of records on these files. Can you please suggest a simple method of finding the number of records. We have two programs written in COBOL which can read the files and can get the number of records but with these tape files are having records around 16-18 million records each and cobol program as such takes long time to read it and generate a report on counts.
We also though of using SORT but with sortout as dummy it does not really help with the execution time. Can there be a way to count just records with less time used.
Thanks
Regards,
Vidya
Vidya
- Robert Sample
- Global Moderator
- Posts: 1900
- Joined: Fri Jun 28, 2013 1:22 am
- Location: Dubuque Iowa
Re: Find the number of records in a Tape File without reading it.
You have a couple of options:
1. Find out which application writes the tape file and check the record counts for the program doing the writes. If the program doesn't write record counts, or you cannot access the application, then see option 2.
2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.
If your tape management system tells you the number of blocks of data, you can estimate the record count by dividing the block size by the record length to get records per block and then multiplying by the number of blocks. Whether or not an auditor would accept such an estimate, however, depends upon your auditors.
1. Find out which application writes the tape file and check the record counts for the program doing the writes. If the program doesn't write record counts, or you cannot access the application, then see option 2.
2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.
Statements like this are worthless to post. What is a "long time" to you may not be a long time to someone else. If you cannot quantify a value, don't put it in your post as it provides no benefit to people reading the post. Furthermore, are you aware that elapsed time for jobs on a z/OS system is HEAVILY dependent upon a great number of factors? These factors include number of address spaces, Workload Manager policy, CPU utilization of the LPAR, channel contention, etc. It is entirely possible that your job could run in half the time if the job class was changed (just as one example) -- but only someone working at your site, such as your site support group, could tell you for sure.cobol program as such takes long time to read it and generate a report on counts.
If your tape management system tells you the number of blocks of data, you can estimate the record count by dividing the block size by the record length to get records per block and then multiplying by the number of blocks. Whether or not an auditor would accept such an estimate, however, depends upon your auditors.
-
- Registered Member
- Posts: 11
- Joined: Thu Jan 28, 2016 9:03 am
Re: Find the number of records in a Tape File without reading it.
First of all thanks for your reply.
How do you decide on the bufferes? Is there some formula for that?
There are so many applications which write these tape files.There is not one application we are looking at. So I shall go for option 2.1. Find out which application writes the tape file and check the record counts for the program doing the writes. If the program doesn't write record counts, or you cannot access the application, then see option 2.
Thanks but reading all the records is the overkill of the process. As the files are huge having 20 million records each. It some times takes more than 6 hours to just read all the records.2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.
How do you decide on the bufferes? Is there some formula for that?
Regards,
Vidya
Vidya
-
- Registered Member
- Posts: 11
- Joined: Thu Jan 28, 2016 9:03 am
Re: Find the number of records in a Tape File without reading it.
The time goes from 3 hours to 6 hours. Mainly dependent on the number of records. I shall check on job class, thanks.Robert Sample wrote:Statements like this are worthless to post. What is a "long time" to you may not be a long time to someone else. If you cannot quantify a value, don't put it in your post as it provides no benefit to people reading the post. Furthermore, are you aware that elapsed time for jobs on a z/OS system is HEAVILY dependent upon a great number of factors? These factors include number of address spaces, Workload Manager policy, CPU utilization of the LPAR, channel contention, etc. It is entirely possible that your job could run in half the time if the job class was changed (just as one example) -- but only someone working at your site, such as your site support group, could tell you for sure.cobol program as such takes long time to read it and generate a report on counts.
I have not explored this option, will check on it.If your tape management system tells you the number of blocks of data, you can estimate the record count by dividing the block size by the record length to get records per block and then multiplying by the number of blocks. Whether or not an auditor would accept such an estimate, however, depends upon your auditors.
Regards,
Vidya
Vidya
- Anuj Dhawan
- Founder
- Posts: 2816
- Joined: Sun Apr 21, 2013 7:40 pm
- Location: Mumbai, India
- Contact:
Re: Find the number of records in a Tape File without reading it.
Tape files are sequential-read, I'm not sure without reading the records, one way or other - how are you going to count them?Vidya Kumari wrote:Thanks but reading all the records is the overkill of the process. As the files are huge having 20 million records each. It some times takes more than 6 hours to just read all the records.2. Read the entire tape file and count the records. This can be done with your SORT product or a custom-written program. If you use a custom-written program, make sure you provide enough buffers (and memory for the buffers) -- typically 50 to 250 would be adequate -- for reading the tape.
Thanks,
Anuj
Disclaimer: My comments on this website are my own and do not represent the opinions or suggestions of any other person or business entity, in any way.
Anuj
Disclaimer: My comments on this website are my own and do not represent the opinions or suggestions of any other person or business entity, in any way.
- Robert Sample
- Global Moderator
- Posts: 1900
- Joined: Fri Jun 28, 2013 1:22 am
- Location: Dubuque Iowa
Re: Find the number of records in a Tape File without reading it.
For your learning, a tape file with 20 million records is NOT "huge". In fact, I would classify it as small to medium size.As the files are huge having 20 million records each.
As far as taking 3 to 6 hours to read a 20-million-record data set, that is HOGWASH! Either there is more going on than you are telling us, or your read JCL is horribly inefficient (such as you accepting the default QSAM buffer allocation), or your job does not have high priority in Workload Manager. For comparison, two weeks ago I needed to read 12 weekly SMF tapes with 2.8 million to 4 million records per tape; the total was 40 million records or so. Including mounting the tapes, reading the SMF data, un-mounting the tapes -- the total elapsed job time was less than 30 minutes. If your job is taking 3 to 6 hours for 20 million records, you need to work with your site support group to find out what is slowing it down as you should be able to read at least 100 million to 500 million (or more) records in 3 to 6 hours.
Create an account or sign in to join the discussion
You need to be a member in order to post a reply
Create an account
Not a member? register to join our community
Members can start their own topics & subscribe to topics
It’s free and only takes a minute