Recovering data

Note

The data formatting or data recovery concepts are identical for all of the products in the ARCAD-Transformer suite.

The data recovery managed in ARCAD Transformer Field has the objective to allow you to carry out the automatic execution of a process from a list of physical file fields. The process will be the same (or nearly the same) for each field on the list.

The process is executed by the programs generated by ARCAD (one program per file). This program declares the file as primary and in update (or as read only, if requested). Each file record is, therefore, read once.

The process to carry out for each field is found in the main part of the program. Each record is updated at the end of the process (if the file was open in update). Unlike other transformer processes, it is possible to use this command from a list of fields of physical files without source.

The data formatting commands can be used to:

fill in the extended part of fields (following an extension of a file’s fields). The previous data having already been copied (in *MAP *DROP, aligned on the left for an alphanumeric field, aligned on the point for a numeric field).
build the new fields (following a field addition in the files). The old data having already been copied (in *MAP *DROP).
carry out an update process (conversion) for the value of certain fields (In this case, there is no modification of the file structure).
carry out an update process (conversion) for the values of certain fields while keeping a trace, in an associated file, of the old field value (or the conversion difference.
automate the starting of a process which must be run for each one of the values in a complete series of physical file fields.
etc...

Using a standard routine written in RPGLE, you can describe (for one copy only) the process to carry out. It is also possible to create an associated file for each file processed.

Working with data recovery sequences

The following two commands allow data recovery:

Generation of PF data field formatting - AGENFMTPGM
Execution of data formatting programs for PF fields - AEXCFMTPGM

These two commands are executed at different times in different environments. It is also necessary to pre-configure them.

Verify or update the impacted fields list

The generation and execution of recovery programs is based on a list of physical file fields to process. This list can:

already have been used by the propagation without modification.
already have been used by the propagation with modification of the PF sources.
be specifically created for generating recovery programs, without going through propagation.

Reference

For more information on the physical file fields, see Field lists.

Important!

You can copy the list, modify its contents (notably LST_CSST and LST_JZSEL2) before using it for generating recovery programs.

Do not hesitate to modify the LST_CSST and LST_JZSEL2 fields, if necessary, to put either other file fields or even the values which will be tested in the standard routine in order to insert a block of code according to the case.

In the list of fields, the fields given in the below table will be used to generate the recovery programs.

Fields used to generate recovery programs
Field	Length/Type	Variable	Contents
LST_JOBJ	10A	$$F1	File field to process
LST_CSST	10A	$$F2	New file field
LST_JZSEL2	30A	$$F3	Third file field
LST_CTYPE	10A	$$TY,$$LG,$$LD	Field type, length and Nb. of decimals
LST_CCPLT	7A	$$FM	Field format (in transformer terms)
LST_CTXT	50A	$$TX	Text of complete field
LST_CTXT	5x10A	$$T1,$$T2,$$T3,$$T4,$$T5	Each of the 5 parts of the text field, with 10 characters (from 1 to 10, 11 to 20, etc.…)
LST_JLIB	10A	$$DM	File record format
LST_JSRCF	10A	$$DF	File name
LST_CATR	10A	$$DT	Type of file (PF,PF38 or PF36)
LST_JXLIB	10A		File library (for FMTFLDDTAX with *FROMLIST)
LST_JZSEL1	10A		Receives the name of the recovery program generated by file.

Execute data formatting programs – AEXCFMTPGM

Objective: To automate the execution of recovery programs, file by file, member by member.

This command can be run:

In the version where the recovery programs are generated.
In the test environment.
In the production environment.
With an online list of libraries, chosen at the start up.
On the AS400 where the programs are generated.
On another AS400 if the generated programs were distributed.

Note

This command does not require an ARCAD Transformer Field license; you just need to have installed ARCAD with an ARCAD Process Manager license.

It is necessary to have the same list of fields for executing the process as for generating the recovery programs.

At the beginning of the process, there is a quick verification for the presence of recovery programs (and possibly any associated files); any anomaly will stop the process from beginning.

In case of failure at the program’s execution on a file, the process continues on to the next part; the non-processed file fields change to status Abn. Once the problem is corrected, it is possible to restart the process in the list while specifying Restart after abnormal end = *YES.

Reference

For more information about AEXCFMTPGM, refer to Executing data formatting programs – AEXCFMTPGM.

Working with associated files for data recovery

Objective: To store information coming from the processed file.

For each record processed, a record can be created in an associated file.

Each field of the processed file has a corresponding field (with the same name) in the associated file. Furthermore, the associated file also includes the primary keys of the processed file in its description. The other fields of the processed file are not present in the associated file.

Utilization cases of an associated file:

To keep an old value of fields before conversion.
To trace the records which will be detected by the recovery program process.
To show the recovery program’s use on the records of the processed file.
etc.…

These associated files are not generally used directly in the application. They can be consulted via SQL, Query or by program.

Specify contents of standard routines for associated files

In this case, a new FRM variable is available:

$$A1 - Indicates the field of the associated file which has the same name in the PF, but which is prefixed by FI2_ in the generated RPGLE program.

Furthermore, to create the record in the associated file, insert the following process to the standard routine:

C		Eval	CreateAssF = '*YES'

The record creation is only made after the processing of all the fields and only if the CreateAssF variable is at *YES.

Also, before writing, the keys of the associated file are loaded from the corresponding fields in the processed file.

Description of primary keys

To obtain a complete associated file, you must know the list of file fields which are considered as primary keys.

The primary keys correspond to the list of file fields which are considered to be the only identifier of the file’s record.

Example

Command N° + Line N° for a command line file.

These primary keys are loaded after running the command Create database links - AUPDFLDDBR. In this loading process, the following are considered as primary keys:

the PF keys if they exist with the key word UNIQUE.
if these are absent, then the keys of the first LF with the key word UNIQUE (by looking at the file xxxL1, xxxL2, then xxxLA, xxxxLB, then the others).
if also absent, then the PF keys if they exist without the keyword UNIQUE.
if also absent, then the keys of the first LF without the key word UNIQUE (by looking at the file xxxL1, xxxL2, then xxxLA, xxxxLB, then the others).
if also absent: No primary key is found.

The method for finding the primary keys does not necessarily correspond to the way your physical and logical files were conceived; it is necessary to verify all the primary keys of all the physical files by:

ADSPFLDDCT – Field Database Repository
Option 35 DSPPFKEY – Display the Primary Keys
Option 32 WRKPFKEY – Work with the Primary Keys

Modify the primary keys if they don’t correspond to what was expected. Your modifications will be retained even if you restart the AUPDFLDDBR command.

Generating data formatting programs – AGENFMTPGM

The AGENFMTPGM command allows you to generate programs for formatting the data of the physical file fields from the database. By formatting, we mean all the processing to be carried out from the PF fields, whether its updates, check processes or even program calls to run for each field.

You can also automate the creation of recordings in an associated file which is also generated.

Once generated, these programs can be executed by the AEXCFMTPGM command based on the same list.

For the generation of programs, the physical files should have already been created in a new format if you want to run the compilation of these recovery programs.

The source programs (and object, if the PGMCPL parameter is at *YES) are generated in the library indicated at the PGMLIB parameter.

These generated programs are named according to the PGMNAM and PGMNUM parameters (for example: FMTF0001, FMTF0002 etc...).

A program is generated for each file present in the field list.

Important!

Run this process in batch to avoid answering questions concerning the destination library, of the source and object, for each program (as can be the case interactively!).

The following table presents the different parameters of the AGENFMTPGM command.

List of impacted fields (LIST)

Enter the name of the list file library that contains the database fields for which you wish to carry out the field formatting.

Sub-routine for process (STDRTN)

Specify here, the name of the standard code routine which will be inserted into the data formatting programs for each field to be processed. To work with routines, use the AWRKSTDRTN command.

This routine must be written in RPGLE.

Propagation group (PRPGRP)

*ANY: All the files present in the fields list are processed (whatever the contents of the field LST_CCPLT – Field format are).

Propagation group + Identification letter : Only those fields with a field format including the specified identification letter are processed.

Work with associated files (ASSFILE)

Specify if you want to work with an associated file for each file processed.

*NO: No associated file is generated.

*YES: An associated file is generated for each file processed. In this case you should fill in the parameters concerning the associated file.

Associated File-Description Library (ASSFDSCLIB)

This parameter is obligatory when you want to work with an associated file.

Enter the name of a library which is not part of your application; if the indicated name doesn’t exist, it will be created automatically.

In this library, the AGENFMTPGM command creates the source of the new associated files having the same name as each processed file. This file will contain, for every processed field, a field having the same name and text but for which the description (Type and Length) can be different. Furthermore, this file will also contain the primary keys of the processed file which allows the correspondence between the original file and the associated file. (The definition of these primary keys is accessible with the command AWRKPFKEY.)

Associated file-Text (ASSFTEXT)