reminder_ChatGPT

地磁気計．2024年６月時点でのデータ処理備忘録．2024-06-06　Y.Okamoto
Geomagnetometer.Data processing reminder as of June 2024.

このところの処理はほぼChatGPT依存の状態で，結構確かなスクリプトを回答してくれる．
The recent processing is almost entirely ChatGPT-dependent, and it answers a pretty solid script.

とりあえず現在の処理に用いているBashシェルスクリプトChatGPT_mv_th.sh
For now, the Bash shell script ChatGPT_mv_th.sh used for the current processing

#!/bin/bash

# assited by ChatGPT on May 2024
# Input and output file names
input_file="2024-06-04.log"
output_file="thinned_data.txt"

# Step 0: Remove unwanted characters (;,=) from the data
tr '=,:XYZ[uT]' ' ' < $input_file > cleaned_data.txt

# Step 1: Extract timestamp and numerical data (assuming space-separated values after cleaning)
awk '{print $1, $2":"$3":"$4, $5, $6, $7}' cleaned_data.txt > extracted_data.txt

# Step 2: Calculate sqrt(x^2 + z^2)
awk '{print $1, $2, sqrt($3*$3 + $5*$5)}' extracted_data.txt > magnitude.txt

# Step 3: Calculate moving average with a window size of 3
awk '{
    window_size = 600
    values[NR] = $3
    sum += $3
    if (NR > window_size) {
        sum -= values[NR - window_size]
    }
    if (NR >= window_size) {
        print $1, $2, sum / window_size
    } else {
        print $1, $2, "N/A"
    }
}' magnitude.txt > moving_average.txt

# Filter the data to only include entries where the seconds are "00"
awk -F " " '$2 ~ /:[0-5][0-9]:00$/' "moving_average.txt" > "thinned_data2.txt"

そして，その後データを午前0時から整列させるのは，process_magnetic_data.awk
And then the data is aligned from midnight by process_magnetic_data.awk

# awk -f process_magnetic_data.awk input_data.txt > output_data.txt

BEGIN {
    FS = " ";
}
{
    time[$2][$1] = $3;
    dates[$1]++;
    all_times[$2]++;
}
END {
    # Print header
    printf "Time";
    for (date in dates) {
        printf "\t%s", date;
    }
    printf "\n";

    # Sort times
    n = asorti(all_times, sorted_times);

    # Print data by sorted time
    for (i = 1; i <= n; i++) {
        t = sorted_times[i];
        printf "%s", t;
        for (date in dates) {
            printf "\t%s", (time[t][date] ? time[t][date] : "");
        }
        printf "\n";
    }
}

これで１日のデータの時刻が午前0時からに整列する．あとはこの出力output_data.txtをcalcで読み込むだけ．
This aligns the time of several daily data starting at midnight. And calc (on Linux) is used to read this output_data.txt.