Read this before you start
Shown below is a selection of software tools developed for the beamline. These apps are all written in Python, hence should work on any platform (Linux, Mac, Windows).
It helps to set up a python environment with the required packages. One suggestion is to use the free conda manager Miniforge.
Here are some instructions:
1. Create a clean environment in Miniforge. To do this, after the installation, start the Miniforge prompt and type: conda create -n myenvironment python==3.xx. Use the flag python=3.11.10, for example.
Note 1: you can check the actively supported python versions on this website: Python versions. Currently, the recommended version is 3.11.10 (the last bug fix), supported until end of 2027.
2. To change to the newly created environment type: activate myenvironment.
Note 2: If you previously uninstalled Anaconda (which is not free for institutional use!), look for the file '.condarc' in your home directory in Windows and edit it prior to running Miniforge, i.e. in the file change 'default' to 'conda-forge' as your only channel. Check it in the prompt by typing: conda config --show channels.
3. For the freshly created environment, open a terminal and install the dependencies using the requirements file included in the app:
pip install -r requirements.txt
Alternatively, use this master file for all apps except EosCross:
Package requirements file
Note 3: If you want to regenerate the provided 'requirements.txt' file , install:
pip install pipreqs
...and run:
pipreqs . --force
4. You can now start the app in the script directory:
python 'filename'.py
Note 4: If you want to use the shortcut, please edit the .bat file!
Menu
ED-XRD
- Detector calibration coefficients
- 2Θ calculator
- File conversion
- NXS Frameviewer & exporter
- EosCross: Joint PT estimation
- Stress (lattice strain) calculator
- PDIndexer troubleshooting
- GSAS-II Guide and troubleshooting
- GSAS-II lst file export (cell params and density)
LVP
Acoustic Emission
ED-XRD
I. Detector calibration between channel and energy
This tool will fit all the peaks produced by the radionuclides Ba133 and Co57 and calibrate a linear and quadratic relationship between channel and energy. You can only use the nxs files obtained at beamline station P61B. This is the first required step when starting a beam time. This tool also produces an input (txt) file for the app NXS2GSAS.
II. 2Θ calculator
A new software tool was developed to obtain the detector positions (2theta) by fitting LaB6. This should be done at the start of a beam time. Please run it in a dedicated Anaconda python environment.
The beamline station P61B uses the NIST LaB6 660c standard with a calibrated lattice parameter of a = 0.415 682 6 ± 0.000 008 nm (95% confidence) to precisely estimate the diffraction angles the detectors are positioned at.
(!) If you require measurements using a different NIST standard (e.g. Si), please discuss with the beamline staff well in advance of your beam time. If you have such a standard material, we request you to bring it along.
Update! v1.3. It is now possible to load an nxs file containing only 1 channel (e.g. if only 1 Ge-detector was used). In addition, for all peaks fitted with the pseudo-Voigt function, an additional txt file is generated containing the full output of the peak fitting, including Chi-squared reduction values. Plus minor bug fixes.
III. File conversion
The file format of the diffraction data saved by the Tango device of the digital analyser for the Ge-detectors is HDF. The file extension is nxs. Below are two tools to convert the diffraction data to readable formats for futher analysis (outside PDindexer).
The first app outputs channel and intensity in two column csv format, without a channel to energy conversion.
The second app NXS2GSAS converts the NXS file to GSAS format for each detector/channel to be used for refinement in GSAS and GSAS-II (since Jan 2022). Additionally, the app writes 2-column txt files for energy and intensity.
-> UPDATE 29.04.2022 - NSX2GSAS no longer requires you to load the channel to energy calibration files. The latest calibration (April 2022) is now included in the script.
As GSAS-II now supports ED-XRD, you can download and install it normally. The converted nxs to GSAS files are compatible. When you import GSAS data to GSAS-II, you must also import an instrument calibration file (see GSAS-II Guide and troubleshooting.
IV. NXS Frameviewer & exporter
This application will help you view large amounts of (ED) diffraction data in a heatmap using the scientifically-accepted colourmap called 'viridis' (see Matplotlib documentation). The displayed data are not modified in any way except by a channel-energy conversion. This is just an alternative presentation of stacked ED-XRD spectra.
The app offers several benefits.
1) Routinely, you can collect a diffraction pattern up to 50 s duration. However, it is also possible to collect data every second for 50 frames. In software, such as PDIndexer, you cannot see the difference because it summates all the frames when loading the nxs file. With this app, you can view each frame as a fucntion of collection time, stacked in a heatmap. You can also open multiple nxs files with each many frames and view them all in a stack.
2) The app can also summate all the frames in multiple nxs files before viewing a stacked sequence. This is very useful if you need good resolution/intensity to see and track the peak positions. You can also observe peak shifts due to deviatoric stress (lattice microstrain) with the detectors in the deformation geometry.
3) You can convert the time scale on the y-axis to temperature if you want to show the evolution of peaks during heating (at regular intervals).
4) You can extract each frame as separate data files in GSAS-II and txt format for point 1 above. Furthermore, you can also do this for point 2 above. Essentially, this is the same as using the NXS2GSAS app, above.
5) The app saves a high-resolution png image of the heat maps when loading the data (you can decide on the dpi quality). NB. due to the dark GUI settings of the app, the saved maps have a black background unfortunately. For use in publications, use any image-editing software (eg. Paint.net) to invert the background to white, with black text. Just make sure to first create selection rectangles of the data area and colour bar, invert this selection, and then invert the image background.
(!) In the app, make sure to update the quadratic equation parameters after a channel-energy calibration of both detectors.
V. EosCross: Joint PT estimation
This tool is very useful to estimate pressure and temperature in situ simultaneously without a thermocouple, via XRD and the equation of state (EoS) of two (or multiple pairs of) materials in the cell assembly. Note, it is best if both materials are in a mixture with suitable proportions. See below for more info. The starting grain size of the powders should be around 1 micron.
Mixing powders of 2 materials. Obtain the Reference Intensity Ratios (RIR) of both materials (e.g. from Match! or some other crystallography software). Divide them by each other to obtain the mixing ratio.
The tool uses Burnman for the calculations and mineral database. Please cite my paper: Farla 2023 and Burnman.
See installation instructions on the official DESY Gitlab website. Link below.
Version 1.3
- New launcher
- Many materials (calibrants) were added in Burnman v1.2. EosCross will now use these instead.
- Over 70 material pairs for joint PT estimation (or P estimation at given temperature) available.
- PT calculations can be exported to txt file.
- Supports batch processing of pressures using all available EoS by loading a file containing the lattice parameters at given temperatures for each pair of materials or single material.
- Uncertainties in unit cell constants/volume and temperature propagated to pressure using a PVT correlation matrix. Uncertainties in the joint PT estimation are also reported.
- Many code changes and optimisations.
V. Stress (lattice strain) calculator
This application is useful for deformation experiments on cylindrical samples in the Aster-15 LVP. Use it to process diffraction data from two detectors to calculate the lattice microstrain Q(hkl) and the hydrostatic pressure dP(hkl) on any sample with cubic, orthorhomic or hexagonal crystal structure.
The calculation only works if one detector is at an azimuthal position (solid angle) of 0 degrees and the other detector at 90 degrees, and the same peaks can be indexed.
Update v 1.1
The main changes are:
- Calculation of the hydrostatic unit cell volume from dP(hkl) for the 3 cases: Cubic, Orthorhombic and Hexagonal structures.
- Fixed the issue for the case when the deformation experiment crosses midnight and the data would not plot correctly.
- Some visual changes in the GUI, updated descriptions, & show the file name at the top to keep track of which file is being processed.
Hint for data plotting: First process all the data files (nxs) for each deformation step and obtain the Q(hkl) and dP(hkl) for each, saved in the newly created Qvalues and dvalues subfolders. Then if you want to plot the results, make sure to match the counter and number of log files (for time) with the data. You can also create a single column csv file with the strain increments. Again, match each strain point with a stress point. When processing the next deformation step, first move the Qvalues and dvalues folders elsewhere, because the app will always create/use these directories.
Possible bug in the data plotter: Plotting Q(hkl) and dP(hkl) against time using incomplete data sets for each hkl is not straightforward. Sometimes you can use a peak, sometimes a peak can disappear, only later to reappear. The script tries to take this into account. Furthermore, it's tricky to know when there is a missing peak so there is a tolerance factor for data sorting, which compares the similar d-spacings (for dPhkl) and sorts the data accordingly. That's why when you plot the data, there can be gaps. If you're missing data for an hkl, maybe this sorting failed, or what you thought was a good peak, is actually not (too inconsistent d-spacings). Anyway, if you feel the missing data can be plotted (e.g. using Matlab or Excel), let me know of this case so I can try to reproduce it.
VII. PDIndexer troubleshooting
First, download the latest version of PDIndexer here: PDIndexer
You also need the latest version of Microsoft .NET 7.0 runtime: dotNET 7.0 runtime
And you also need a recent version of the HDF5 libraries: HDF5 pre-built binaries
Simply install these above and PDIndexer should load.
- PDIndexer does not require administrative priviliges, it will install in the /user directory.
- You must use regional settings in MS Windows where the decimal point is used (e.g. UK), rather than the comma (as is common in Germany).
- After an update, I STRONGLY recommend to reset the crystals (File -> Revert crystals to the initial state) and clear the registry (Option -> Clear Registry (check and restart)).
In case you do not have any EDXRD data from P61B yet, you can load a dummy datafile:
Download datafile
You will also need the channel to energy calibrations before you can open any data files (.nxs). Below are the calibrations from May 2023.
CH0 (detector 2)
- a0: -27.704634233000
- a1: 40.190729052025
- a2: -0.000014869539
- a0: -0.221663132172
- a1: 40.158842651421
- a2: -0.000014072325
VIII. GSAS-II Guide and troubleshooting
We recommend you to refine your ED-XRD data using GSAS-II in order to obtain the unit cell volume and density of the phases at high PT conditions in the LVP. You can download a guide on how to do this.
From GSAS-II version 5625 (and latest version), both Gaussian and Lorentzian parameters are included in the peak fitting. This is particularly useful if peak shapes are a bit non-Gaussian. In fact, the new method approximates a pseudo-Voigt model, which is preferrable. For the latest versions you need a different instrument parameters file when importing ED-XRD powder data than when using the old version with only Gaussian parameters. To get started, download the correct instrument parameter file for the version of GSAS2 you plan to use. You need this file to open your first powder file (e.g. LaB6). Note that data collected in your beam time cannot use these files directly.
First, run a refinement on each ED-XRD LaB6 data file (channel 0 & channel 1). Keep the parameters A and C fixed at zero (0), while B can be 0.0005 initially; these are the Gaussian terms. Also keep the parameters X, Y and Z fixed at zero (0); these are the Lorentzian terms. Notably, in the latest version of GSAS2, the parameters XE, YE, ZE, and WE are ignored (whatever values they may have), because a channel-energy conversion was already done. For angle calibration, refinements should include 2theta, but it should hardly change from the value(s) obtained with the other software (above). If it does, something is wrong! Save/export the new refined instrument parameter file (for each channel) and use those to refine your ED-XRD powder data, keeping 2theta fixed. Hint: for your data, typically only refine the background parameters, the B parameter (Gaussian) and the Y parameter (Lorentzian), not the others peak shape parameters (A,C,X,Z), because they will cause problems (errors) in the refinements!
UPDATE!On 27 Nov. 2024, I received an explanation of the parameter file and functionality of GSAS-II for ED-XRD analysis. Please consider these words carefully:
The parameters XE, YE, ZE & WE are the MCA calibration parameters for channel number to energy; they are found in the data header. They are only shown in the Instrument Parameters for information. They are currently not refinable - they are (in my understanding) established by the MCA manufacturer; they only got used when the EDX data was imported to convert channel no. to energy. After that it doesn't matter what their values are. The 2-theta value is refinable by using a standard with known lattice parameter. A, B, C are Gaussian instrument broadening terms & X, Y, Z are Lorentzian instrument broadening terms (hard to know a situation where they could be non-zero). Sample broadening in GSAS-II is always handled in the Data tab for the phase; those are phase/histogram specific & have possibly complex forms. The sample broadening is almost invariably Lorentzian so trying to use B for that isn't appropriate anyway. Given the low resolution of most EDX patterns, finding sample broadening will be difficult unless the broadening is pretty extreme. NB: most sample broadening is due to microstrain not crystallite size (except for nanomaterials - submicron in size). As for gonio. radius - it could be of use for determining a sample displacement even for EDX patterns - possible future development.
As for the instprm file - the order doesn't matter & could change with python version as it is a rendering of a dictionary. -- Robert Von Dreele.
IX. GSAS-II lst file export (cell params and density)
This utility is useful for quickly extracting the unit cell parameters and density of all phases from the GSAS-II lst file saved after each refinement. It is most effective when you created a GSAS-II project with multiple imported powder data. The lst file generated by GSAS-II takes the filename of the project, so each time you refine an ED-XRD pattern, the refined unit cell parameters and density of each phase are overwritten in this file. Use the app to extract this information from the lst file between refinements of the powder data in the GSAS-II project and append each result into a txt file (leave the app open). The txt file can be edited e.g. using MS Excel to plot the data, or can be used for further data processing (e.g. to fit an EoS to the PVT data of the phase).
LVP
I. Sample pressure calculator
The calculators are based on these pressure calibrations:
II. Aster-15 Logviewer (MATLAB)
If you want to quickly and interactively visualize the press log data saved by the Aster-15 LVP, as well as the logged heating data if the AC heating system was used, then you can run this MATLAB script.
Now you can also download the executable for the Aster-15 logviewer, below. To run it, you also need to download and install the correct MATLAB runtime version 9.12 (R2022a). You can get it from here.
III. Decompression profile maker for Aster-15
There are cases where a linear decompression curve is not ideal. For example, it could lead to a decompression blow out or the sample will be under tension, causing cracking. This app will generate an exponential decay curve for decompression and you have the option to change the shape between a linear and purely exponential function. This curve can be saved to an xml file, which can be imported into the Aster-15 LVP software. The software accepts up to 19 steps in the pressure profile.
Extract the zip file, and execute 'profilemaker.exe'
Acoustic Emission
I. Extraction and processing AE data (Mistras DTA file)
For Acoustic Emissions detection experiments in the LVP, the Mistras/GMA acquisition software AEwin is used. It is now possible to extract your AE data from the proprietary file format *.dta by using MistrasDTA:
Using the above module, I created a python app to extract waveforms and view and filter events based on their AE characteristics. Download link below:
Below are the older versions of the DTA extraction tool, if you still find them useful...
Version 1.2: Correction applied the precision on the time of arrival (TOA) of the hits/events, which should be 7 decimals (i.e. 250 ns resolution). Cleaned up a few other lines in the code.
Version 1.3: Added a notification dialogue when no events can be found based on the chosen arrival time precision (python script only!)
II. Interactive DTA waveviewer
For Acoustic Emissions detection experiments in the LVP, the Mistras/GMA acquisition software AEwin is used. Using this custom software, you can view your AE data directly by opening the proprietary file format *.dta. Like the above app 'DTA extract', the waveviewer apps also make use of MistrasDTA:
You can download both the python file and the compiled executable below:
If you only wish to see the waveforms that belong to events, you can use the modified app below. You also have the option to exclude a bad channel.
Version 1.1: Added a notification dialogue when no events can be found based on the chosen arrival time precision (python script only!)
