This page was generated from
docs/Examples/Fitting_Fermi_Diads/Example1c_HORIBA_Calibration/Step4_Stitch_Outputs_Together.ipynb.
Interactive online version:
.
4. Stitch outputs together
Once you have fitted the weak, medium and dense diads, stitch them all together, and apply the Ne correction model
[1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import joblib
import DiadFit as pf
pf.__version__
[1]:
'0.0.77'
[2]:
# Load in settings, paths, file extensions etc. that you saved in step 1
meta_path, spectra_path, spectra_filetype, prefix, str_prefix, spectra_file_ext, meta_file_ext, TruPower=pf.get_settings()
Load in specra, and combine
[3]:
import os
df2=pf.merge_fit_files(path=os.getcwd())
First, get the metadata for this day
[ ]:
# Put the common string you can in all your Ne lines here
diad_meta=pf.get_files(path=meta_path,
file_ext=spectra_file_ext, exclude_str=['Ne', 'FG', 'Cap','window', '_CRR', 'laser'],
sort=True)
diad_meta[0]
'10mol_FID_R1.txt'
Now get all the important information out of the metadata files
[11]:
meta=pf.loop_convert_datastamp_to_metadata(path=spectra_path,
files=diad_meta, creation=False,
modification=True)
meta
[11]:
| filename | date | Month | Day | power (mW) | Int_time (s) | accumulations | Mag (X) | duration | 24hr_time | sec since midnight | Spectral Center | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10mol_FIE_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 1:46:39 | 6399 | NaN |
| 0 | 10mol_FIE_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 1:49:45 | 6585 | NaN |
| 0 | 10mol_FIE_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 1:53:30 | 6810 | NaN |
| 0 | 10mol_FID_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:12:49 | 7969 | NaN |
| 0 | 1mol_FIB_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:31:39 | 9099 | NaN |
| 0 | 1mol_FIB_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:42:55 | 9775 | NaN |
| 0 | 1mol_FIB_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:51:31 | 10291 | NaN |
| 0 | 10mol_FID_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:5:15 | 7515 | NaN |
| 0 | 10mol_FID_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:8:46 | 7726 | NaN |
| 0 | 1mol_FIA_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:15:6 | 11706 | NaN |
| 0 | 1mol_FIA_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:22:7 | 12127 | NaN |
| 0 | 1mol_FIA_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:28:32 | 12512 | NaN |
| 0 | LL8_404a_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:49:24 | 13764 | NaN |
| 0 | LL8_404a_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:55:13 | 14113 | NaN |
| 0 | 4mol_FID_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:17:56 | 15476 | NaN |
| 0 | LL8_404a_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:1:40 | 14500 | NaN |
| 0 | 4mol_FID_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:23:29 | 15809 | NaN |
| 0 | 4mol_FID_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:27:51 | 16071 | NaN |
| 0 | LL8_626_MI_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:44:17 | 17057 | NaN |
| 0 | LL8_626_MI_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 4:50:16 | 17416 | NaN |
| 0 | LL8_626_MI_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 5:1:9 | 18069 | NaN |
| 0 | LL8_622B_R2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 5:25:16 | 19516 | NaN |
| 0 | LL8_622B_R1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 5:25:27 | 19527 | NaN |
| 0 | AutoCalibration - Laser 532.05 nm - Grating 18... | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 5:41:21 | 20481 | NaN |
| 0 | LL8_622B_R3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 5:51:6 | 21066 | NaN |
| 0 | LL8_612_a_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:13:34 | 22414 | NaN |
| 0 | LL8_612_a_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:20:50 | 22850 | NaN |
| 0 | LL8_612_c_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:28:25 | 23305 | NaN |
| 0 | LL8_612_c_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:35:22 | 23722 | NaN |
| 0 | LL8_612_c_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:47:29 | 24449 | NaN |
| 0 | LL8_612_a_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 6:6:55 | 22015 | NaN |
| 0 | LL8_615_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:11:19 | 25879 | NaN |
| 0 | LL8_615_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:18:11 | 26291 | NaN |
| 0 | LL8_631_a_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:34:55 | 27295 | NaN |
| 0 | LL8_631_a_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:41:30 | 27690 | NaN |
| 0 | LL8_631_a_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:47:30 | 28050 | NaN |
| 0 | LL8_615_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:4:48 | 25488 | NaN |
| 0 | LL8_631_b_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 7:54:48 | 28488 | NaN |
| 0 | LL8_631_b_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 8:10:20 | 29420 | NaN |
| 0 | LL8_631_b_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 8:3:40 | 29020 | NaN |
| 0 | LL8_624_r1.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 8:48:35 | 31715 | NaN |
| 0 | LL8_624_r2.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 8:54:37 | 32077 | NaN |
| 0 | LL8_624_r3.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:0:34 | 32434 | NaN |
| 0 | 10mol_FIE_R1_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:25:3 | 33903 | NaN |
| 0 | 10mol_FIE_R2_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:28:16 | 34096 | NaN |
| 0 | 10mol_FIE_R3_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:31:32 | 34292 | NaN |
| 0 | 10mol_FID_R1_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:35:26 | 34526 | NaN |
| 0 | 10mol_FID_R2_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:38:51 | 34731 | NaN |
| 0 | 10mol_FID_R3_FIN.txt | January 1, 2020 | January | 1 | NaN | NaN | NaN | NaN | NaN | 9:43:2 | 34982 | NaN |
[7]:
meta['filename'].unique()
[7]:
array(['10mol_FID_R1_CRR_DiadFit.txt', '10mol_FID_R1_FIN_CRR_DiadFit.txt',
'10mol_FID_R2_CRR_DiadFit.txt', '10mol_FID_R2_FIN_CRR_DiadFit.txt',
'10mol_FID_R3_CRR_DiadFit.txt', '1mol_FIA_R3_CRR_DiadFit.txt',
'1mol_FIB_R1_CRR_DiadFit.txt', '10mol_FIE_R3_FIN_CRR_DiadFit.txt',
'1mol_FIA_R1_CRR_DiadFit.txt', '1mol_FIA_R2_CRR_DiadFit.txt',
'10mol_FIE_R1_FIN_CRR_DiadFit.txt',
'10mol_FIE_R2_FIN_CRR_DiadFit.txt',
'10mol_FID_R3_FIN_CRR_DiadFit.txt', '10mol_FIE_R2_CRR_DiadFit.txt',
'10mol_FIE_R1_CRR_DiadFit.txt', '10mol_FIE_R3_CRR_DiadFit.txt',
'LL8_612_a_r1_CRR_DiadFit.txt', 'LL8_404a_r2_CRR_DiadFit.txt',
'LL8_404a_r1_CRR_DiadFit.txt', 'LL8_612_a_r2_CRR_DiadFit.txt',
'4mol_FID_R3_CRR_DiadFit.txt', '1mol_FIB_R3_CRR_DiadFit.txt',
'LL8_612_a_r3_CRR_DiadFit.txt', 'LL8_404a_r3_CRR_DiadFit.txt',
'1mol_FIB_R2_CRR_DiadFit.txt', '4mol_FID_R1_CRR_DiadFit.txt',
'4mol_FID_R2_CRR_DiadFit.txt', 'LL8_612_c_r1_CRR_DiadFit.txt',
'LL8_622B_R3_CRR_DiadFit.txt', 'LL8_624_r3_CRR_DiadFit.txt',
'LL8_615_r2_CRR_DiadFit.txt', 'LL8_612_c_r2_CRR_DiadFit.txt',
'LL8_612_c_r3_CRR_DiadFit.txt', 'LL8_624_r1_CRR_DiadFit.txt',
'LL8_615_r1_CRR_DiadFit.txt', 'LL8_622B_R1_CRR_DiadFit.txt',
'LL8_622B_R2_CRR_DiadFit.txt', 'LL8_615_r3_CRR_DiadFit.txt',
'LL8_626_MI_r1_CRR_DiadFit.txt', 'LL8_624_r2_CRR_DiadFit.txt',
'LL8_631_b_r1_CRR_DiadFit.txt', 'LL8_631_a_r3_CRR_DiadFit.txt',
'LL8_631_b_r2_CRR_DiadFit.txt', 'LL8_631_b_r3_CRR_DiadFit.txt',
'LL8_626_MI_r2_CRR_DiadFit.txt', 'LL8_631_a_r1_CRR_DiadFit.txt',
'LL8_626_MI_r3_CRR_DiadFit.txt', 'LL8_631_a_r2_CRR_DiadFit.txt',
'10mol_FIE_R1.txt', '10mol_FIE_R2.txt', '10mol_FIE_R3.txt',
'10mol_FID_R3.txt', '1mol_FIB_R1.txt', '1mol_FIB_R2.txt',
'1mol_FIB_R3.txt', '10mol_FID_R1.txt', '10mol_FID_R2.txt',
'1mol_FIA_R1.txt', '1mol_FIA_R2.txt', '1mol_FIA_R3.txt',
'LL8_404a_r1.txt', 'LL8_404a_r2.txt', '4mol_FID_R1.txt',
'LL8_404a_r3.txt', '4mol_FID_R2.txt', '4mol_FID_R3.txt',
'LL8_626_MI_r1.txt', 'LL8_626_MI_r2.txt', 'LL8_626_MI_r3.txt',
'LL8_622B_R2.txt', 'LL8_622B_R1.txt',
'AutoCalibration - Laser 532.05 nm - Grating 1800 - Coeff 0.00429475 PASS-postcrash.txt',
'LL8_622B_R3.txt', 'LL8_612_a_r2.txt', 'LL8_612_a_r3.txt',
'LL8_612_c_r1.txt', 'LL8_612_c_r2.txt', 'LL8_612_c_r3.txt',
'LL8_612_a_r1.txt', 'LL8_615_r2.txt', 'LL8_615_r3.txt',
'LL8_631_a_r1.txt', 'LL8_631_a_r2.txt', 'LL8_631_a_r3.txt',
'LL8_615_r1.txt', 'LL8_631_b_r1.txt', 'LL8_631_b_r3.txt',
'LL8_631_b_r2.txt', 'LL8_624_r1.txt', 'LL8_624_r2.txt',
'LL8_624_r3.txt', '10mol_FIE_R1_FIN.txt', '10mol_FIE_R2_FIN.txt',
'10mol_FIE_R3_FIN.txt', '10mol_FID_R1_FIN.txt',
'10mol_FID_R2_FIN.txt', '10mol_FID_R3_FIN.txt'], dtype=object)
Then get a simple file name you can stitch with the spectra
[12]:
file_m=pf.extracting_filenames_generic(names=meta['filename'],
file_ext=spectra_file_ext)
# Checks they are all unique
file_m[0]
good job, no duplicate file names
[12]:
'10mol_FIE_R1'
Now get filename from spectra into same form as metadata
[13]:
# Remove these to get the pure file name
file_s=pf.extracting_filenames_generic(
names=df2['filename'].reset_index(drop=True),
file_ext=spectra_file_ext)
# Remove the cosmic rays, as doesnt have it in the metatdata
file_s = np.char.replace(file_s.astype(str), "_CRR_DiadFit", "")
print(file_s[0])
good job, no duplicate file names
1mol_FIA_R1
[14]:
# Combining them together, if the form above matches, e.g. checking one doesnt have a suffix and the other one does
meta['filename']=file_m
df2['filename']=file_s
df_combo=pd.merge(df2, meta, on='filename').reset_index(drop=True)
df_combo.head()
[14]:
| filename | Splitting | Split_σ | Diad1_Combofit_Cent | Diad1_cent_err | Diad1_Combofit_Height | Diad1_Voigt_Cent | Diad1_Voigt_Area | Diad1_Voigt_Sigma | Diad1_Residual | ... | Month | Day | power (mW) | Int_time (s) | accumulations | Mag (X) | duration | 24hr_time | sec since midnight | Spectral Center | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1mol_FIA_R1 | 102.767219 | 0.044476 | 1286.315198 | 0.042497 | 2562.841337 | 1286.315198 | 5446.273124 | 0.844234 | 13.421863 | ... | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:15:6 | 11706 | NaN |
| 1 | 1mol_FIA_R2 | 102.688849 | 0.022089 | 1286.484421 | 0.019341 | 2137.380237 | 1286.484421 | 4842.994026 | 0.982687 | 14.282855 | ... | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:22:7 | 12127 | NaN |
| 2 | 1mol_FIA_R3 | 102.740191 | 0.031933 | 1286.538044 | 0.028257 | 1941.160783 | 1286.538044 | 4777.388422 | 1.006703 | 18.429721 | ... | January | 1 | NaN | NaN | NaN | NaN | NaN | 3:28:32 | 12512 | NaN |
| 3 | 1mol_FIB_R2 | 102.840122 | 0.231069 | 1286.056444 | 0.230759 | 94165.069563 | 1286.056444 | 53955.485426 | 0.257576 | 19.771037 | ... | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:42:55 | 9775 | NaN |
| 4 | 1mol_FIB_R3 | 102.762320 | 0.075155 | 1286.305769 | 0.071756 | 1225.734745 | 1286.305769 | 2754.391123 | 0.860494 | 20.665060 | ... | January | 1 | NaN | NaN | NaN | NaN | NaN | 2:51:31 | 10291 | NaN |
5 rows × 50 columns
If you have secondary phases, now is the time to merge those in
[15]:
df_combo_sec_phase=pf.merge_in_carb_SO2(
df_combo=df_combo, file1_name='Carb_Peak_fits.xlsx', file2_name='SO2_Peak_fits.xlsx',
prefix=prefix, str_prefix=prefix_str, file_ext=spectra_file_ext)
#df_combo_sec_phase.to_excel('FI_fitting_w_sec_phases.xlsx', index=False)
If you have SO2, now is the time to calculate the molar proportion of SO2
[17]:
if 'Peak_Area_SO2' in df_combo_sec_phase:
SO2_ratio=pf.calculate_CO2_SO2_ratio(peak_area_SO2=df_combo_sec_phase['Peak_Area_SO2'],
peak_area_diad1=df_combo_sec_phase['Diad1_Voigt_Area'], peak_area_diad2=df_combo_sec_phase['Diad2_Voigt_Area'],
wavelength=532.067, T_K=37+273.15,efficiency_SO2=1, efficiency_CO2=0.5,
sigma_SO2=4.03, sigma_CO2_v1=0.8, sigma_CO2_v2=1.23)
df_combo_sec_phase['SO2_mol_ratio']=SO2_ratio['SO2']