reminder_ChatGPT

Seis 移動平均計算と描画　2025-02-15
Seis Moving Average Calculation and Drawing

既存の地震計データ(64Hzサンプリングを4HZに間引きしたもの)で移動平均処理を行い，特に冬季の風による脈動を抑える．
Moving average processing of existing seismograph data (64 Hz sampling thinned to 4Hz) to suppress wind-induced microseisms, especially in winter.

元の図（オリジナル）
Original

chatGPTによるスクリプトの最初のものからの修正．行端にスペースではなくキャリッジ・リターン処理を行う．
これをしないと，Drum描画処理で配列読み込み時にエラーとなる．

Fixes from the first script by chatGPT today. Carriage returns are now used at the end of lines instead of spaces.
If this is not done, an error will occur when loading arrays in the Drum drawing program.

English Version of awk script

# Moving Averaged for Seis data by ChatGPT, Y.Okamoto 2025-02-15
# awk -v N=16 -f M_AVE.awk input.txt > output.txt

# Get the window size for moving average
BEGIN {
    if (N <= 0) {
        print "Error: N must be greater than 0"
        exit 1
    }
}

# Process each line of data
{
    for (i = 1; i <= 6; i++) {
        # Store data for each channel
        data[NR, i] = $i

        # Calculate cumulative sum for moving average
        sum[i] += $i

        # If window size is exceeded, subtract the oldest data
        if (NR > N) {
            sum[i] -= data[NR-N, i]
        }

        # Calculate moving average (output starts from the Nth line)
        if (NR >= N) {
            ave = sum[i] / N
            output = sprintf("%04d", ave)
        } else {
            # Output original data for lines less than N
            output = sprintf("%s", $i)
        }

        # Add space except for the last column
        if (i < 6) {
            printf "%s ", output
        } else {
            printf "%s\n", output
        }
    }
}

N=8(cutoff=2sec) と 16(4sec)を試したが，後者が有効．当然ながら，inputなどはrenameして用いる．
We tried N=8 (cutoff=2sec) and 16 (4sec), the latter being more effective. Naturally, inputs, etc., should be renamed.

Usages;
awk -v N=8 -f M_AVE.awk input.txt > output.txt
awk -v N=16 -f M_AVE.awk input.txt > output.txt

移動平均後
Moving Averaged

地磁気計．2024年６月時点でのデータ処理備忘録．2024-06-06　Y.Okamoto
Geomagnetometer.Data processing reminder as of June 2024.

このところの処理はほぼChatGPT依存の状態で，結構確かなスクリプトを回答してくれる．
The recent processing is almost entirely ChatGPT-dependent, and it answers a pretty solid script.

とりあえず現在の処理に用いているBashシェルスクリプトChatGPT_mv_th.sh
For now, the Bash shell script ChatGPT_mv_th.sh used for the current processing

#!/bin/bash

# assited by ChatGPT on May 2024
# Input and output file names
input_file="2024-06-04.log"
output_file="thinned_data.txt"

# Step 0: Remove unwanted characters (;,=) from the data
tr '=,:XYZ[uT]' ' ' < $input_file > cleaned_data.txt

# Step 1: Extract timestamp and numerical data (assuming space-separated values after cleaning)
awk '{print $1, $2":"$3":"$4, $5, $6, $7}' cleaned_data.txt > extracted_data.txt

# Step 2: Calculate sqrt(x^2 + z^2)
awk '{print $1, $2, sqrt($3*$3 + $5*$5)}' extracted_data.txt > magnitude.txt

# Step 3: Calculate moving average with a window size of 3
awk '{
    window_size = 600
    values[NR] = $3
    sum += $3
    if (NR > window_size) {
        sum -= values[NR - window_size]
    }
    if (NR >= window_size) {
        print $1, $2, sum / window_size
    } else {
        print $1, $2, "N/A"
    }
}' magnitude.txt > moving_average.txt

# Filter the data to only include entries where the seconds are "00"
awk -F " " '$2 ~ /:[0-5][0-9]:00$/' "moving_average.txt" > "thinned_data2.txt"

そして，その後データを午前0時から整列させるのは，process_magnetic_data.awk
And then the data is aligned from midnight by process_magnetic_data.awk

# awk -f process_magnetic_data.awk input_data.txt > output_data.txt

BEGIN {
    FS = " ";
}
{
    time[$2][$1] = $3;
    dates[$1]++;
    all_times[$2]++;
}
END {
    # Print header
    printf "Time";
    for (date in dates) {
        printf "\t%s", date;
    }
    printf "\n";

    # Sort times
    n = asorti(all_times, sorted_times);

    # Print data by sorted time
    for (i = 1; i <= n; i++) {
        t = sorted_times[i];
        printf "%s", t;
        for (date in dates) {
            printf "\t%s", (time[t][date] ? time[t][date] : "");
        }
        printf "\n";
    }
}

これで１日のデータの時刻が午前0時からに整列する．あとはこの出力output_data.txtをcalcで読み込むだけ．
This aligns the time of several daily data starting at midnight. And calc (on Linux) is used to read this output_data.txt.