\chapter The ENCODE programs

\subrub 
Description of Programs
%---------------------%

The ENCODE programs are a package of four programs which create the
condensed (PDF-1) database files and index files needed by Powder Suite. 
The first program, PDF1, creates the PDF-1 data files from the full (PDF-2)
JCPDS-ICDD database along with an index to the PDF-1 and PDF-2 database files.
The second program, ADDPDF1, is used to add PDF-1 format entries to the 
PDF-1 files written by program PDF1. These entries are easier to prepare
than PDF-2 entries. The third program, WRDI, is used to create the d-space 
index files which are used for rapid searching of the database by peak 
position and the fourth program, WRCHEM, is used to create the elemental 
composition index files which are used for rapid searching of the database by 
chemistry.

The programs must be run in sequence, but the second program, ADDPDF1,
may be omitted unless it is desired to add additional PDF-1 entries into the
local version of the JCPDS-ICDD database.

\subsubrub Program PDF1

The first program, PDF1, reads the PDF-2 database and creates a condensed 
subset of the original database called the PDF-1 database. The PDF-1 
database consists of two files: NAME.DIRECT, which contains the 
chemical formula, chemical and mineral names; DI.LIBRARY, which
contains the d-spaces (peak positions) and intensities for the peaks.
Also written at this time is an index to the database, INDEX.DIRECT and
an index to the various subfiles in the database, MISC.BITMAP.

Some users may wish to add their own entries to the database. If a file
of AIDS*83 format entries (the format used in the PDF-2 database) is 
prepared, these entries may also be indexed by the PDF1 program and will 
be accessible by the Powder Suite programs identically with all other 
PDF-1 entries. Program LOGIC as both PDF-1 and PDF-2 entries. A copy of the 
PDF-2 database licensed by the ICDD is required to run this program.

\subsubrub Program ADDPDF1

The second program, ADDPDF1, is used to add PDF-1 format entries to the 
PDF-1 files written by program PDF1 (NAME.DIRECT, DI.LIBRARY, MISC.BITMAP
and INDEX.DIRECT). These entries contain much less information than is 
required in a PDF-2 entry and are easier to prepare. This program does 
not require access to a PDF-2 database and thus may be used at locations
where only the PDF-1 format database has been licensed.

Use of this program is optional and should be skipped if there are no 
PDF-1 entries to add. It is also possible to run ADDPDF1 more than once, 
adding additional entries with each subsequent run. Note that program 
PDF1 creates the PDF-1 database files from the PDF-2 database, while 
program ADDPDF1 adds new entries to the PDF-1 files. Thus, each time 
that program PDF1 is run, all PDF-1 entries must be added anew to the 
PDF-1 files using ADDPDF1.

\subsubrub Program WRDI

Program WRDI is used to create the d-space index files than are used
for rapid searching of the database by peak position. These files
consist of a single bit mapped to each database entry, the bit is set
if the entry has a peak above the intensity threshold in the specified
location. These files are also referred to as bitmap indices for this 
reason. 

Two types of index files may be written by this program:
\TAB  
  (1)	a file that indexes the location of three strongest peaks in 
	each entry, DSP3S.BITMAP
\TAB
  (2)	a set of files that index the location of all peaks above an 
	intensity threshold of |xx|\%, DSP|xx|.BITMAP. The number of
	bitmap files that are created are at the descretion of the
	person who runs this program and the amount of available disk
	space. At a minimum files should be written with 0\% and 70\% 
	thresholds. It is recommended that a 30\% file be written,
	and that any other files be written at increments of 10\%.

This program must be run after ADDPDF1 or PDF1 to update the index 
files.

\subsubrub Program WRCHEM

Program WRCHEM is used to create the elemental composition 
index file, file CHEM.BITMAP, that is used for rapid searching of 
the database by chemistry. This file consists of a single bit mapped 
to each database entry, the bit is set if the entry contains the 
specified element. This file is sometimes referred to as the chemistry 
bitmap index for this reason.

This program must be run after ADDPDF1 or PDF1 to update the index 
files.

\subrub 
Running the Programs
%------------------%

While the programs may be run interactively, they are fairly slow and
cumbersome and will be most convenient to run as batch jobs. The
following sections discuss the DCL logical names needed by each program
and the input needed. Following the discussion, an example batch input
file is shown.

\subsubrub Program PDF1

Program PDF1 requires the definition of several logical names for input
and output:
\par{\parskip 0pt\leftskip 1.25 in
\TAB      |PS_PDF2_DISK|\   The location of the PDF-2 disk file
\TAB      |PS_PDF2_CDROM|\  The location of the PDF-2 CDROM
\TAB      |PS_PDF2_LOCAL|\  The file with local PDF-2 additions to the database
\TAB      |PS_PDF2_INDEX|\  The location of the PDF-2 support files
\TAB      |PS_JCPDS|\       The directory where the PDF-1 files will be written
\TAB      |PS_BITMAP|\      The directory where MISC.BITMAP will be written
\parskip 0pt\par}

To run the program, one must define where the PDF-2 files will be located
by either defining |PS_PDF2_DISK| or |PS_PDF2_CDROM| (not both). The default 
name for the PDF-2 file if on disk is JCPDS.DIRECT, if the file has some
other name, the full file name should be specified, otherwise only the
device and directory need be used. The file must be a 80-byte fixed record
length file. Note that the disk file can be read via DECNET from another
node. The default name for the CDROM is file name PDF2.DAT on device 
DUB0:; if both are true, then |PS_PDF2_DISK| does not need to be defined.
If the file is on another device, that device must be specified. Note
that DECNET access to the PDF2 is not allowed, however sharing of the
CDROM in a cluster may be possible.

Input is required to decide how to process "deleted" patterns in the 
PDF-2 database. While some patterns in the JCPDS-ICDD database are 
removed by the editors completely, some are flagged as "deleted" and
a reference is given to the entries which specify the later entries 
which superceed these "deleted" patterns. Three options are available
for processing of these "deleted" patterns. 
\TAB 
   (1)	The patterns may be omitted completely from the PDF-1 
	database. If this is done, it is not possible to ever
	see the PDF-2 entries in program LOGIC. To do this enter
	a "N" on the first line of input.

\TAB
   (2)	The patterns may have their chemical formula listed in 
	the PDF-1 database but not their peak positions (d-spaces 
	and intensities). If this is done it is possible to locate
	"deleted" entries based on chemical searches, but not on
	peak position searches and the PDF-2 entries may be 
	displayed in program LOGIC. To do this enter a "Y" on 
	the first line of input and a "N" on the second.

\TAB
   (3)	The patterns may have both their chemical formula and peak 
	positions listed in the PDF-1 database. If this is done,
	"deleted" patterns are processed largely the same as 
	the other patterns. To do this enter a "Y" on 
	the first line of input and a "Y" on the second.

The program also requires as input the expected sizes of the output
NAMES.DIRECT and DI.LIBRARY files, in blocks. These numbers are used to 
reserve a block of space for each file. Note that there must be 
sufficient space on the disk to allocate files of this size, or the
program will not run. All unused space will be released when the
program ends. So, it is better to overestimate the space requirements, 
than underestimate them.

The three PDF-2 support files needed by PDF1 are |DISP.INIT|, 
|CODENS|xx|.DAT|, and |VAXPDF|xx|.BIN|, where xx is a set number
(|40|, |41|, ...). The location of these files is defined using 
logical name |PS_PDF2_INDEX|. These files are produced by the 
JCPDS-ICDD for use with the PDF-2 database and are updated with 
each release of the database. If the files are updated, all three
files should be updated together.

If local additions are to be supplied to the program, an 80-byte fixed
record length file must be created with AIDS*83 format entries (see
below). The entries must have card numbers above 99 and must have a set
number of zero. Thus the entries must have be given numbers between 101
(0,0101) and 9999 (0,9999). The entries must be placed in the file 
sorted by order of increasing card number.

Sample batch input for program PDF1:

\par{\parskip 0pt\leftskip 0.5 in
	|$ SET VER|\par
	|$!  DCL procedure to create the PDF-1 files from the PDF_2 file|\par
	|$ define	PS_pdf2_index	adisk:[toby.ps.pdf2]|\par
	|$ define	PS_pdf2_disk	ESVAX::PUBLIC:[TOBY.SET39]|\par
	|$ define	PS_pdf2_local	local.pdf2|\par
	|$ define	PS_JCPDS	adisk:[toby.ps.pdf1]|\par
	|$ define	PS_bitmap	adisk:[toby.ps.pdf1]|\par
	|$ run adisk:[toby.ps.encode]pdf1|\par
	|Y|\par
	|N|\par
	|10000|\par
	|20000|\par
	|$ EXIT|\par
\parskip 0pt\par}

The following procedure is an example of a method which may be used 
to create a fixed length record file when creating the Local PDF-2 
database.

\par{\parskip 0pt\leftskip 0.5 in
	|$ SET VER|\par
	|$!  DCL procedure to create a local PDF-2 file|\par
	|$ create fixrec.fdl|\par
	|FILE|\par
	|	ALLOCATION		0|\par
	|	BEST_TRY_CONTIGUOUS	yes|\par
	|	EXTENSION		0|\par
	|	ORGANIZATION		sequential|\par
	|RECORD|\par
	|	BLOCK_SPAN		yes|\par
	|	CARRIAGE_CONTROL	none|\par
	|	FORMAT			fixed|\par
	|	SIZE			80|\par
	|$ convert/fdl=fixrec/pad   sys$input  adisk:[toby.ps.pdf2]local.jcpds|\par
	|                                                      M           P    P000101X1|\par
	|                                                    368.14       0.00  P000101X4|\par
	| O                                                                     P000101X5|\par
	|Naphthylbismuth Dioxide (BHT)                                       P  P000101X6|\par
	|C10 H7 Bi O2                                                           P000101X7|\par
	|C10 H7 Bi O2                                                           P000101X8|\par
	|ANCHAM  10  475 1938 Foo, Bar et al.                                   P000101X9|\par
	|MoKa1 0.709         F        DD         X 16.0                         P000101XF|\par
	|X                      BB                                              P000101XG|\par
	|20.0000000             9.90000 18             3.40000  5              1P000101XI|\par
	|3.22000  3             3.15000  3             2.98000  3              2P000101XI|\par
	|2.67000 10             2.06000  3             1.93000  5              3P000101XI|\par
	|1.88000 10             1.61000  5             1.57000  3              4P000101XI|\par
	|1.52000  3             1.33000  3             1.24000  3              5P000101XI|\par
	|1.19000  5                                                            6P000101XI|\par
	|                   EGS 85/ 1/13  0  0 08 84/12/13                      P000101XK|\par
	|$ del fixrec.fdl;|\par
	|$ exit|\par
\parskip 0pt\par}

\subsubrub Program ADDPDF1

Program ADDPDF1 requires the definition of two logical names for input
and output:
\par{\parskip 0pt\leftskip 1.25 in
\TAB      |PS_JCPDS|\       The directory where the PDF-1 files will be read 
			  and written
\TAB      |PS_BITMAP|\      The directory where MISC.BITMAP will be read and
			  written
\parskip 0pt\par}

The program reads files NAMES.DIRECT, DI.LIBRARY, JCPDS.INDEX and 
MISC.INDEX and writes new files in the same directories. 

The program requires as input the expected sizes of the output
NAMES.DIRECT and DI.LIBRARY files, in blocks. These numbers are used to 
reserve a block of space for each file. Note that there must be 
sufficient space on the disk to allocate files of this size, or the
program will not run. All unused space will be released when the
program ends. So, it is better to overestimate the space requirements, 
than underestimate them.

The PDF-1 additions must also be supplied to the program as a set
of lines for each entry. Card numbers must be above 99, as before,
but there is no upper limit for the entry numbers. It is recommended,
however, that numbers not exceed 9999, so that local entries do not
replace entries in the database. The entries must be input sorted in
order of increasing card number. If desired, this program may be used
to enter new PDF-1 entries for existing PDF-2 entries, thus overriding
the entries in the ICDD-JCPDS database.

A PDF-1 entry consists of the following free-format lines of input. 

{\par\parskip 2pt
\TAB	A) card number
\TAB	B) chemical formula
\TAB	C) chemical name
\TAB	D) number of peaks, reference intensity (I/Ic)
\TAB	E) pairs of d-spaces and intensities (one or more pairs per line)
\TAB	F) subfile flags 
\parskip 0pt\par}

The chemical formula should consist of element symbols followed by 
an element count. If a mineral name is to be specified, it should 
follow the chemical name, separated by a slash. Up to 200 peaks
may be specified for an entry. Peak positions should be given as 
d-spaces (in \AA) and peak intensities are relative and will be scaled
by the program so that the strongest peak is 100\%.
The subfile flags may be one or more of the following numbers: 
{\par\parskip 2pt
\TAB	 1  Inorganic
\TAB	 2  Organic
\TAB	 3  Common Phases
\TAB	 4  Minerals
\TAB	 5  Alloy/Metal/Intermetallic
\TAB	 6  NIST (NBS)
\TAB	 7  Forensics
\TAB	 8  Zeolites
\TAB	 9  Deleted patterns
\TAB	10  Local patterns (always set)
\parskip 0pt\par}

All subfile flag values must be on one line, separated by commas.

Sample batch input for program ADDPDF1:

\par{\parskip 0pt\leftskip 0.5 in
	|$ SET VER|\par
	|$ set def adisk:[toby.ps.PDF1]|\par
	|$!  DCL procedure to add PDF-1 entries to the main PDF-1 files|\par
	|$ define	PS_JCPDS	adisk:[toby.ps.pdf1]|\par
	|$ define	PS_bitmap	adisk:[toby.ps.pdf1]|\par
	|$ run adisk:[toby.ps.encode]addpdf1|\par
	|15000|\par
	|25000|\par
	|201|\par
	|Al2 O2|\par
	|gamma-Alumina (Toby)|\par
	|5|\par
	|20.0, 10|\par
	|10.0, 50|\par
	|5.0, 100|\par
	|4.0, 40,   3.0, 20|\par
	|1,3,4|\par
	|202|\par
	|Al2 O2|\par
	|delta-Alumina (BHT)|\par
	|2|\par
	|20.0, 50|\par
	|10.0, 50|\par
	|1,3,4,9|\par
	|-1|\par
	|$ PURGE	adisk:[toby.ps.pdf1]*.*|\par
	|$ EXIT|\par
\parskip 0pt\par}

\subsubrub Programs WRCHEM and WRDI

Programs and WRCHEM and WRDI require the definition of a logical name for
input and a second one for output:

\par{\parskip 0pt\leftskip 1.25 in
\TAB       |PS_JCPDS|\       The directory where the PDF-1 files will be read 
\TAB       |PS_BITMAP|\      The directory where bitmap files will be written
\parskip 0pt\par}

Programs and WRCHEM and WRDI are most commonly run together in a single
batch run. No input is required for WRCHEM. Program WRDI requires a set
of input values that determines which bitmaps will be created by the 
programs. A value between 0 and 99 causes a index file to be created
for all peaks with intensity greater than or equal to the specified 
value (where the strongest peak in the pattern is scaled to 100.) A 
value of -1 causes the program to write an index file to the "three 
strongest peaks."

Sample batch input for program WRDI and WRCHEM (in a single run):

\par{\parskip 0pt\leftskip 0.5 in
	|$ SET VER|\par
	|$! DCL procedure to create the JCPDS index files from the PDF-1|\par
	|$ define	PS_JCPDS	adisk:[toby.ps.pdf1]|\par
	|$ define	PS_bitmap	adisk:[toby.ps.pdf1]|\par
	|$ run adisk:[toby.ps.encode]wrdi|\par
	|-1|\par
	|70|\par
	|30|\par
	|00|\par
	|$ run adisk:[toby.ps.encode]wrchem|\par
	|$ PURGE	adisk:[toby.ps.pdf1]*.*|\par
	|$ EXIT|\par
\parskip 0pt\par}
