地磁気計.2024年6月時点でのデータ処理備忘録.2024-06-06 Y.Okamoto
Geomagnetometer.Data processing reminder as of June 2024.
このところの処理はほぼChatGPT依存の状態で,結構確かなスクリプトを回答してくれる.
The recent processing is almost entirely ChatGPT-dependent, and it answers a pretty solid script.
とりあえず現在の処理に用いているBashシェルスクリプトChatGPT_mv_th.sh
For now, the Bash shell script ChatGPT_mv_th.sh used for the current processing
#!/bin/bash
# assited by ChatGPT on May 2024
# Input and output file names
input_file="2024-06-04.log"
output_file="thinned_data.txt"
# Step 0: Remove unwanted characters (;,=) from the data
tr '=,:XYZ[uT]' ' ' < $input_file > cleaned_data.txt
# Step 1: Extract timestamp and numerical data (assuming space-separated values after cleaning)
awk '{print $1, $2":"$3":"$4, $5, $6, $7}' cleaned_data.txt > extracted_data.txt
# Step 2: Calculate sqrt(x^2 + z^2)
awk '{print $1, $2, sqrt($3*$3 + $5*$5)}' extracted_data.txt > magnitude.txt
# Step 3: Calculate moving average with a window size of 3
awk '{
window_size = 600
values[NR] = $3
sum += $3
if (NR > window_size) {
sum -= values[NR - window_size]
}
if (NR >= window_size) {
print $1, $2, sum / window_size
} else {
print $1, $2, "N/A"
}
}' magnitude.txt > moving_average.txt
# Filter the data to only include entries where the seconds are "00"
awk -F " " '$2 ~ /:[0-5][0-9]:00$/' "moving_average.txt" > "thinned_data2.txt"
そして,その後データを午前0時から整列させるのは,process_magnetic_data.awk
And then the data is aligned from midnight by process_magnetic_data.awk
# awk -f process_magnetic_data.awk input_data.txt > output_data.txt
BEGIN {
FS = " ";
}
{
time[$2][$1] = $3;
dates[$1]++;
all_times[$2]++;
}
END {
# Print header
printf "Time";
for (date in dates) {
printf "\t%s", date;
}
printf "\n";
# Sort times
n = asorti(all_times, sorted_times);
# Print data by sorted time
for (i = 1; i <= n; i++) {
t = sorted_times[i];
printf "%s", t;
for (date in dates) {
printf "\t%s", (time[t][date] ? time[t][date] : "");
}
printf "\n";
}
}
これで1日のデータの時刻が午前0時からに整列する.あとはこの出力output_data.txtをcalcで読み込むだけ.
This aligns the time of several daily data starting at midnight. And calc (on Linux) is used to read this output_data.txt.
![](graph.png)
びCopyright(c) by Y.Okamoto 2024, All rights reserved.