Reading EBCDIC file with multiple structure

[MJames1030]

Hi,

I'm using the cobrix library on databricks to read EBCDIC file. I have now a copybook with multiples structure. When I read the EBCDIC file all the data of the file are in the structure of the first structure.

Here my option :
spark.read.format("cobol").option("record_length", "100").option("pedantic", "true").option("ebcdic_code_page", "cp1047").option("drop_value_fillers", "false").option("drop_group_fillers", "false").

I have also tried with the option : is_record_sequence = true

Here an extract of the file.

STRUCTURE DE RECORD DU COM-SET 03 00010000
01 REC-TYPE1. 00020000
03 T1-RECTYPE PIC X(02). 00030000
03 T1-MODYEAR PIC X(04). 00040000
03 T1-CODEMOD PIC X(06). 00050000
03 T1-LANGUE PIC X(02). 00060000
03 T1-SEQNUMB PIC X(03). 00070000
03 FILLER PIC X(13). 00080000
03 T1-CODEMODDESC PIC X(40). 00090000
03 T1-FILLER PIC X(02). 00100000
03 T1-MARQUE PIC X(01). 00110000
03 T1-PROCCOD PIC X(01). 00120000
03 T1-STARTDAT PIC X(07). 00130000
03 T1-EXPIRDAT PIC X(07). 00140000
03 T1-MODDATE PIC X(07). 00150000
03 T1-MODTIME PIC X(04). 00160000
03 FILLER PIC X(01). 00161000
00162000
01 REC-TYPE2. 00163000
03 T2-RECTYPE PIC X(02). 00164000
03 T2-MODYEAR PIC X(04). 00165000
03 T2-CLASS PIC X(02). 00166000
03 T2-LANGUE PIC X(02). 00167000
03 T2-PRNUMB PIC X(03). 00168000
03 T2-SEQNUMB PIC X(03). 00169000
03 FILLER PIC X(14). 00170000
03 T2-PRNRDESC PIC X(40). 00180000
03 FILLER PIC X(02). 00190000
03 T2-MARQUE PIC X(01). 00200000
03 T2-PROCCOD PIC X(01). 00210000
03 T2-STARTDAT PIC X(07). 00220000
03 T2-EXPIRDAT PIC X(07). 00230000
03 T2-MODDATE PIC X(07). 00240000
03 T2-MODTIME PIC X(04). 00250000
03 FILLER PIC X(01). 00260000
00270000
01 REC-TYPE3. 00280000
03 T3-RECTYPE PIC X(02). 00290000
03 T3-MODYEAR PIC X(04). 00300000
03 T3-CLASS PIC X(02). 00310000
03 T3-LANGUE PIC X(02). 00320000
03 T3-PACKAGE PIC X(03). 00330000
03 T3-SEQNUMB PIC X(03). 00340000
03 FILLER PIC X(14). 00350000
03 T3-PACKDESC PIC X(40). 00360000
03 FILLER PIC X(02). 00370000
03 T3-BRAND PIC X(01). 00380000
03 T3-PROCCOD PIC X(01). 00390000
03 T3-STARTDAT PIC X(07). 00400000
03 T3-EXPIRDAT PIC X(07). 00410000
03 T3-MODDATE PIC X(07). 00420000
03 T3-MODTIME PIC X(04). 00430000
03 FILLER PIC X(01). 00440000

There are 15 structures like this

Do you know who I can solve this ?

Thank you in advance,
Jamal

[yruslan]

If the first record looks good, but rest of records are not, the most likely is the cobybook is not aligned with the record size.

Please, try the latest version 2.6.11, since a bug was fixed that caused Cobrix to ignore .option("record_length", "100") in certain circumstances.

If that won't help, you can use .option("debug", "true) to debug and determine the correct record size.