Crystals User Guide

CRYSTALS contains routines for performing most parts of a structure analysis with the exceptions of diffractometer data pre-processing, analytical absorption corrections and direct methods. Excellent programs for direct methods are available from groups specialising in those areas, and initial data processing tend to be rather site specific.

The code has been written to be efficient in the maths and data handling, and robust and informative in the user interaction. There are no intrinsic limits on the numbers of atoms, reflections or refined parameters. Only one array need be redimensioned to increase the size of matrix handled. The current limit is 512 variables in a single block. However, such a task would probably be ill-considered. CRYSTALS has been used to refine large molecules, such as Insulin.

Plain language keyed free format input, unified command syntax for batch and interactive modes, parameter defaults, data checking, plain text error messages, 'data subroutine' libraries, data libaries, data-base management and an optional interogative user interface.

Extensive documentation, including a manual, this Guide, user modifiable HELP files, interactive querying for instructions and keywords, examples and frequently asked questions.

Simple syntax for refering to individual atoms, and to groups of atoms, with or without the application of symmetry.

Data reduction of photographic and diffractometer data, including empirical absorption corrections. Application of decay curves, rejection of systematic absences, merging of equivalent reflections with special treatment of outliers, processing of both X-ray and neutron diffraction data.

Structural parameter editor for modifying individual parameters, atoms or groups of atoms. Modifications available include: addition, subtraction, multiplication, division of parameters; application of matrices to coodinates; renaming, deletion, reordering generation of atoms; application of symmetry operators; selection by type or parameter value; interconversion between U[aniso] and U[equiv]

Structural analysis and model building. Best line and plane through groups of atoms, projection of other atoms onto these planes, forcing planarity onto parts of structure. TLS analysis of groups of atoms, forcing U[aniso] for groups to conform to rigid body motion. Comparison between a group of atoms and another group, a geometric figure or structure from the literature. Model a group of atoms onto another group, figure or structure. Geometric hydrogen and other atom placing.

Reflections to be skipped or included are selected individually or by tests. Testable parameters include Fo, Fc, sigma, I/sigma(I), phase, sin theta, index, zone, etc. The test conditions can be changed at any time without loss of data.

Restrained refinement. Distances and angles restrained to numeric values or their mean values; components of U[aniso] along bonds; planarity of groups; sums and averages of parameters; shift limiting restraints; and user defined FORTRAN-like expressions given as data.

Highly flexible least-squares process. User can define up to 256 blocks per cycle. Blocks can be as small as one parameter per block, one atom per block, or full matrix, or any intermediate combination of parameters. Any atom parameter shifts (X's, U's, Occ) can be equivalenced, linked, made to ride. Parameters can be redefined as sums and diferences. Atoms can be refined as rigid groups. Shift multipliers can be applied, the current matrix can be re-used, structure factors need only be calculated for partial structures. Data for up to 8 twin components can be refined. Refinable parameters include overall scale, overall Uiso, Dummy Uiso, Larson extinction, Rogers polarity, Flack enantiopole, batch scales, layer scales, twin components, x, y, z, Occ, Uiso, Uaniso.

15 weighting schemes are available, including statistical, unit, Hughes Cruickshank and Dunitz schemes. Weighting optimisation via a Chebychev polynomial, checking for outliers by the method of Tukey and Prince, can be used to to get 'correct' parameter esds. Analysis of residuals by Fo, sintheta, index, class, parity.

Fourier maps are computed with any number of points at any resolution along x, y, z. Map types are Fo, Fc, Fo-Fc, 2Fo-Fc, Fo**2, Fc**2. Maps can be weighted, including Sim weighting. Peak search gives indication of peak quality. Automatic Fourier refinement, peak rejection and molecule assembly. Lineprinter and VDU plots of atoms and peaks.

Comprehensive distance and angle calculation, with selection by atom type, interatomic distance, connectivity, inter molecular, intramolecular. Symmetry included. Esds from the full variance-covariance matrix. Torsion angles, planes, lines, TLS (with bond correction), principal axes of Uaniso.

Publication listings of atomic parameters, distances, angles, torsion angles, reflections.

An over all scheme for a well behaved structure analaysis is:

  1. Select the best crystal you can find. This is your primary
  data source, and determines the quality and ease of the rest
  of the procedure. Complex mathematics is a poor substitute 
  for good experimental practices.
  
  2. Collect the best data you can. Collect some (at least 10%)
  equivalent reflections for an indication of the data quality.
  Collect systematic absences for glide planes  and screw axes.
  Collect systematic absences for centred cells out to theta =
  theta(max)/2.
  
  3. Preprocess the reflection data with some local software to 
  get a file of h, k, l, I, sigma(I), and setting angles if 
  these are needed for some geometry dependent absorption 
  correction.
  
  4. Use CRYSTALS to do data reduction, and submit a Direct 
  Methods job with MULTAN, SHELXS, or SIR.
  
  5. If Direct Methods fail, check the data, space group, 
  crystal density, data reduction process. If these are all 
  OK, use CRYSTALS or SHELXS to compute a Patterson  map. 
  Seek help.
  
  6. Input the trial structure to CRYSTALS and perform Fourier 
  refinement. IT IS IMPORTANT to realize that though least 
  squares refinement is often used early in a refinement, there
  is no guarantee that it will converge, and in fact the results
  may actually be mis-leading. Proceed to refinement with care.
  
  7. Compare the final results critically with your expectations.
  An unusual effect in part of the structure, even if of no 
  direct interest for the current investigation, may be 
  symptomatic of a faulty refinement.
 

The CRYSTALS system can be used in many ways. The program can be used in batch and interactively - the best solution probably being a combination of both modes. Each CRYSTALS job may be made independent of preceeding jobs by inputing all numeric data from card image files at the start of each job, or the analysis may be carried out as a cumulative process, with the results of each stage being held in a binary data file. Data is inserted into this file by reading it from a card image file, from the terminal, or as a result of a succesful calculation. Data in this file can always be listed to a terminal or printer, or output as a card image file for archiving and communication with other programs.

Commands to CRYSTALS may include some numeric data as part of the command, but in general they require that the data has been input previously. The crystallographic routines check for the existence of the necessary data before they start computations, and generate error messages if the data is not available. Batch jobs consisting of multiple commands, which fail through lack of some resource (time, disk space), may usually be restarted after the last succesful command.

The numeric data for CRYSTALS is collected into groups of related parameters, called LISTS. The data is both stored on the disk and represented to the user as LISTS. Thus LIST 5 contains all the refinable parameters, so that the model under refinement is often refered to as 'LIST 5'. In their character representatons, these LISTS may either be typed directly into a running CRYSTALS system, or put into character files using some method provided by the computers operating system.

The following table, which can be displayed online from a running CRYSTALS program by typing \MANUAL LISTS, gives the function of most lists.

  Lists marked * cannot be input by the User
  
  List  Volume   Type of data
  Number  Name
          
    1    VOL2    Cell parameters
    2    VOL2    Unit cell symmetry
    3    VOL2    Atomic scattering factors
    4    VOL5    Weighting parameters
    5    VOL4    Atomic parameters
    6    VOL3    Reflection data
   10*   VOL6(4) Peak coordinates from Fourier
   11    VOL5    Least squares matrix
   12    VOL5    Refinement directives
   13    VOL2    Crystal and collection data
   14    VOL6    Fourier directives
   16    VOL5    General Restraint instructions
   17    VOL5    Special Restraint instructions
   22    VOL5    Refinement directives in internal format
   23    VOL5    Structure factor calculation control list
   24*   VOL5    Least squares shift list
   25    VOL5    Twin component operators
   26*   VOL5    Constraints in internal format
   27    VOL3    Diffractometer scales
   28    VOL5    Reflection condition list
   29    VOL2    Contents of asymmetric unit 
                 and elemental properties
   31    VOL2    Cell parameter E.S.D.'s
   33*   VOL5    Internal - Refinement control
 

This table, available via \MANUAL COMMANDS, lists the commands currently implemented in CRYSTALS.

               Available INSTRUCTIONS
  
  Introduction to the system
  
  \FINISH       \ text     \TITLE        \COMMANDS     
  \HIDE         \SHOW      \LOG          \USE          
  \PAUSE        \TYPE      \MANUAL       \DISK      
  \PURGE        \RELEASE   \SUMMARY      \FOREIGN
  
  Initial data input
  
  \SPACEGROUP  \COMPOSITION \QUICKSTART
  
  Reflection handling
  
  \HKLI         \LP        \SYSTEMATIC   \SORT         
  \MERGE        \REORDER   \REFLECTIONS  \DIFABS
  
  Atomic and structural parameters
  
  \EDIT         \CONVERT   \HYDROGENS    \REGULARISE
  
  Structure factors and refinement
  
  \SFLS         \LIST 22   \LIST 26      \SPECIAL      
  \CHECK        \TRIAL     \ANALYSE      \WEIGHT       
  \CLEAR
  
  Fourier maps
  
  \FOURIER      \SLANT     \PEAKS        \COLLECT      
  \REGROUP
  
  Analysis of results
  
  \AXES         \DISTANCES \MOLAX        \TORSION      
  \PARAMETERS   \ANISO
 

By default, interactive running of CRYSTALS starts up in 'SCRIPT' mode, in which the user is offered short menues of possible actions, and prompted for required data. The action of the program, including the questions posed, the defaults offered and the actions taken, is under the joint control of the user and of data files called SCRIPTS. However, they are only data, and the users are encouraged to modify and extend them. Changing these files CANNOT corrupt the CRYSTALS program, though they can of course be made to corrupt data.

The master script, CONTROL, calls other scripts in a tree like way, and is perhaps useful for beginners. However, if the user knows the name and function of a sub-script, he can enter it directly from command level with the command '\SCRIPT scriptname'. This list (June 1988) shows what scripts are available.

  CONTROL        CRYSHELP       DATRED         DEVELOP        
  DIFABS         DIFFOUR        DIRECT         EDEMAP         
  EDLIST1        EDLIST29       EDPEAKS        ENDLSQ         
  EXECUTE        FINISH         FOURDEVEL      FOURREF        
  GENERAL        HELP           INCAD4         INFORN         
  INHKLI         INITIALLSQ     INLIST1        INLIST12       
  INLIST13       INLIST14       INLIST2        INLIST23       
  INLIST29       INLIST3        INLIST5        INPUT          
  INSPACE        LISTS          LOOK           MITHRIL        
  MODIFY         MULTAN         NEWMODEL       NEWSTRUCTURE   
  PATTERSON      QUESTIONS      REDFOR         REFINE         
  RENAME         SHELXS         SIR            SOLVE          
  SPAWN          STARTUP        TYPE           USE            
  WEIGHTS        
 

CRYSTALS can be run in a mixed mode. In this mode, the user initiates an interactive job and then via the USE instruction directs CRYSTALS to take input from the nominated file. If the file ends with \USE CONTROL, control is returned to the interactive terminal, where the user may issue the next instruction, which indeed could be another \USE. USE files may call other files, to a depth of 5, though any file already being USEd may not be called from another in-use USE file (the data stack may not be used recursively). Note that the file being USEd need not be a complete LIST or INSTRUCTION block. It can for example be a single line of data, such as a scattering factor to be included into a LIST 3 from a library. The USE instruction is also available in batch mode.

The user has files of the type 'atom.cu' containing the scattering factor for the given atom with the anomalous component for Cu radiation.

       C.CU contains:
       
  SCATT C 0 0 1.930 12.72 1.878 28.65 1.57 .59 .37 65.03 0.25
 

The CRYSTALS commands are:

 
  \LIST 3
  READ NSCAT=3
  \USE H.CU
  \USE C.CU
  SCATT F 0 0 3.304 11.265 3.017 4.665 1.357 .338 .836 27.99
  END
 

The user has in his file store four files: his DISK file, his initial data (LISTINPUT), his compressed reflections (PLANES), and his current parameters (ATOMS). His aim is to compute structure factors.

During a given run of CRYSTALS, the program MUST be linked to a direct access file (DISK) holding the data-base for the current structure. This file provides the mechanism for data transfer between different stages of the task, one stage of which may well be the input of the data itself. On sites where users are allocated sufficient file store to preserve their DISK file between jobs, the DISK also provides a mechanism for communicating between jobs. In fact, if the DISK is preserved, CRYSTALS makes no distinction between successive tasks run in the same job and the same tasks run in successive jobs. Instructions exist in CRYSTALS for 'punching' out files containing reflections in various formats, including a compressed format, and the structure coordinates. In most systems, the 'punched' files will remain in the filestore as character (ASCII) files in a format suitable for direct re-input to a subsequent job. If therefore, the user keeps a file of his initial data, such as cell parameters and scattering factors, and terminates each job by outputting his current parameters as a 'punched' file, he is in a position to create a scratch DISK file for the duration of a job, and then delete it at the end. Even if the user is able to preserve his DISK between jobs, it is prudent to punch out the parameters from time to time and keep these in a safe place, together with the initial data and the compressed reflections.

CRYSTALS plain-language output to a number of devices. Depending on the installation and mode of use, one or more of these devices may be a VDU. In the list below, the default file extension is given in parenthesis.

CRYSTALS uses a number of work files in the course of most calculations. Of particular interest are the sequential files. Their use in initial reflection processing is explained in the Manual. The system manager will provide a suitable JCL procedure or program environment to assign these files to scratch files. The user should however be aware of the existence of these files on systems where file store quotas operate. Similar watch should be kept on output files if these are attributed to the user while waiting to be listed.

In general, however, on a well regulated system the user should normally only be concerned with his primary source of reflection data, his initial crystallographic data, his DISK file and those files he creates for controlling individual jobs.

The help facility in CRYSTALS is neither a computer-aided learning facility nor a substitute for the manuals, but rather an aide-memoire for the interactive user.

The instruction \HELP HELP lists those facilities for which some descriptive help is provided in the form of hints or tips.

The instruction \MANUAL INDEX explains where definitions of LISTS and INSTRUCTIONS can be found in the manual. The manual is indexed by Volume, Chapter, Section and Page number.

If he can remember which INSTRUCTION he needs to use, the \COMMAND facility will tell the user what DIRECTIVES, KEYWORDS and PARAMETERS are available for that INSTRUCTION. Where appropriate, it will also show default values. In addition, it also gives the values of some data-base and system parameters mainly of use to programmers.

The 'query' facility enables the user to interrogate the data-base. Lines beginning with '?' are not interpreted as data, but generate brief output from the database. The user curious about the workings of the input processing routines should read the volume DATABASE in the manual.

The manual is extensive, but its completeness means it is the best place to look for detailed instructions and more unusual commands. The manual occasionally refers to 'default' values. These values are set at the discretion of the system manager, who may well wish to change them for specfic reasons. The file COMMANDS should be available to show the values currently installed, and the user can always use the \COMMAND facility to obtain unequivocal values.

The manual defines the main system INSTRUCTIONS:

  \FINISH \TITLE     \ comment   \SET     \OPEN      
  \HELP   \USE       \PAUSE      \RELEASE \TYPE    
  \MANUAL    \$
 

The instruction 'SET MONITOR OFF' might well be the first instruction in a file to be USEd to recreate a DISK file, and 'SET MONITOR ON' the penultimate. They would inhibit the listing of information the user knows to be valid, and restore it before returning to the main control stream. Comment lines, which begin \<space>, can be inserted anywhere in the control channel data except in the body of a LIST 6 (reflections), and should be included in all files that are to be archived.

The introduction also defines the LIST input and output instructions:

  \LIST     \PRINT     \PUNCH     \SUMMARY
 

The \LIST instruction is used to introduce a discrete group of data, called a LIST, into CRYSTALS, which validates it and then stores it on the DISK. Lists can be introduced to the DISK in any order and whenever the user likes, provided that the lists required for a calculation are on the DISK when that calculation is asked for. The program checks that the necessary lists are present before the calculation is started, and warns the user of omissions. All lists stored on the DISK can be PRINTed, though the format is rarely useful for publication purposes (for that, see RESULTS). These listings do however show the value associated with every element of the LIST, including those taken by default or modified by a calculation. \SUMMARY produces a summary of the required LIST. The \PUNCH instruction produces character files suitable for archiving or re-input to CRYSTALS.

The DISK house-keeping can be altered by the instructions:-

  \DISK     \PURGE
 

When the DISK is created, special code is run to allocate space in the file to hold two indices, the DISK index and the CURRENT index. An entry is put into the DISK index for each LIST subsequently added to the DISK. In general, when a new LIST is input or generated by calculation or modification, this new list is put after the last existing list on the DISK, and its address entered into the DISK index. This index will now contain two or more entries for lists of the given type, and so that subsequent calculations will know which one to use, the address of the most recent is inserted into the CURRENT index. Thus the CURRENT index is of fixed length, with one slot for each possible list type, and the DISK index is of variable length with a slot for each list present on the DISK. The DISK index itself consists of fixed length blocks. In the event that a block becomes full, a new block is chained to the old one. This new block will be after the last list which was written. The indexing of the DISK, and hence the addressing of the lists, may become undefined if the last DISK index block extends beyond the physical end of the file. Neither the DISK nor the CURRENT index is updated until a list is safely written to the DISK, so that in the event of a system failure or a crash of the program, no evidence remains of the incomplete list, i.e. the integrity of the DISK is preserved. Exceptions to this are during access to the reflection list (LIST 6) and the normal matrix (LIST 11). Because these lists are so large, the system will either try to update or overwrite them. If a job fails during an update, the list cannot be accessed again until a job has been run to complete the update. For example, if a job fails during a round of least squares, there may be some Fc values present for the current atomic parameters, and some left from a previous calculation. The user will not be able to access LIST 6 for, say, a FOURIER synthesis until he has recomputed structure factors for all the reflections.

Because new lists are usually added to the end of the DISK, the user has on his DISK a short term archive and record of the calculations he has done. In order to return to some previous state of the calculation, he merely has to enter the address of the list he wishes to use into the CURRENT index. To simplify this, each list is given a SERIAL number when it is written to the DISK. The serial number for LIST 5 (parameters) is usually displayed whenever the list is written to the DISK, and the user can RESET a LIST to a previous SERIAL if it is prefered to the current list. Other lists can be monitored by the instructions \SET WATCH and \SET LIST. Eventually there may be so many versions of lists on the DISK that it risks becomming full. The user can PURGE his DISK of old lists, only the current or spacially marked ones remaining. If there are some particularly important early versions, the user can RETAIN them during the purge. A purge rewrites the required files at the beginning of the DISK, releasing space for new ones. It does not physically shorten the file. During a purge the DISK is in a very delicate state, possibly with some of the vital information only existing in the computer memory. A machine or job failure during a purge could well produce a DISK file that is unusable. The system may during certain types of machine or calculation failure mark lists as being in error, or alter their write/overwrite ststus. Such lists cannot normally be accessed for calculations, but if the user is convinced he understands why the flags were set, he can if he wishes reset them with MARK or USAGE. Thus the status of LIST 6 may reasonably be reset if a failure occurs during the calculation of structure factors, since the Fo values will still be valid, and new Fc values can be recomputed. It would be quite unreasonable to reset the status of a LIST 6 marked as in error during data reduction. The EXTEND directive in \DISK allows the user to control the growth of the disk file.