File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change 1+ ---
2+ title : 2025/12/12 TREM Lite 偵測重複觸發問題
3+ authors : [yuyu1015]
4+ tags : [yuyu1015, rca]
5+ ---
6+
7+ import SeverityBadge from " @site/src/components/SeverityBadge" ;
8+
9+ <SeverityBadge level = " medium" />
10+
11+ ---
12+
13+ ## 事件概述
14+
15+ ** 發生時間:** 2025/12/12 04:24:28 UTC+8
16+ ** 觸發事件:** 地震後
17+ ** 問題描述:** TREM Lite 出現偵測重複觸發的問題
18+ ** 根本原因:** 因資料不一致導致系統異常
19+
20+ <iframe
21+ width = " 560"
22+ height = " 315"
23+ src = " https://www.youtube.com/embed/UI9hKzP17LE"
24+ frameborder = " 0"
25+ allow = " accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
26+ allowfullscreen
27+ ></iframe >
28+
29+ ## 根本原因分析
30+
31+ ### 問題根源
32+
33+ Balance Server 上的 Nginx 使用了以下負載平衡設定:
34+
35+ ``` bash
36+ upstream core_servers {
37+ server tnn.exptech.dev:80;
38+ server tpe.exptech.dev:80;
39+
40+ hash $remote_addr consistent;
41+ }
42+ ```
43+
44+ ### 問題說明
45+
46+ ` hash $remote_addr consistent; ` 會根據用戶端的 IP 位址進行一致性雜湊(Consistent Hashing),將來自同一 IP 的請求分配到特定的 Core Server。
47+
48+ ** 問題點:**
49+
50+ - 當網路不穩定時,用戶端 IP 可能發生變化
51+ - 導致資料來源在兩個 upstream(` tnn.exptech.dev ` 和 ` tpe.exptech.dev ` )之間來回切換
52+ - 造成資料不一致,進而引發 TREM Lite 偵測重複觸發的問題
53+
54+ ## 解決方案
55+
56+ ### 修改後的設定
57+
58+ 將 Nginx upstream 設定改為以下配置:
59+
60+ ``` bash
61+ upstream core_servers {
62+ server tnn.exptech.dev:80 max_fails=1 fail_timeout=10s;
63+ server tpe.exptech.dev:80 backup;
64+ }
65+ ```
66+
67+ ### 改進說明
68+
69+ 1 . ** 主伺服器設定:** ` tnn.exptech.dev ` 設為主伺服器,並設定故障檢測參數
70+
71+ - ` max_fails=1 ` :允許最多 1 次失敗
72+ - ` fail_timeout=10s ` :故障後 10 秒內不再嘗試該伺服器
73+
74+ 2 . ** 備援伺服器設定:** ` tpe.exptech.dev ` 設為備援伺服器(backup)
75+
76+ - 僅在主伺服器不可用時才會使用
77+
78+ 3 . ** 移除一致性雜湊:** 避免因 IP 變化導致的資料來源切換問題
79+
80+ ## 後續觀察
81+
82+ 經觀察後,故障應該已經排除。建議持續監控系統狀態,確保問題不再發生。
Load Diff This file was deleted.
Original file line number Diff line number Diff line change @@ -3,21 +3,3 @@ yuyu1015:
33 title : Full-Stack Engineer (全端工程師)
44 url : https://github.com/whes1015
55 image_url : https://github.com/whes1015.png
6-
7- yoyo0901 :
8- name : Yoyo0901
9- title : Flutter Engineer (Flutter工程師)
10- url : https://github.com/Yoyochou0901
11- image_url : https://github.com/Yoyochou0901.png
12-
13- cstrikeasia :
14- name : Miyashooooo
15- title : Front-End Engineer (前端工程師)
16- url : https://github.com/cstrikeasia
17- image_url : https://github.com/cstrikeasia.png
18-
19- timyaya8732 :
20- name : Timyaya8732
21- title : Data Analyst Engineer (資料分析工程師)
22- url : https://github.com/timyaya
23- image_url : https://github.com/timyaya.png
Original file line number Diff line number Diff line change @@ -3,47 +3,7 @@ yuyu1015:
33 permalink : /yuyu1015
44 description : YuYu1015
55
6- yoyo0901 :
7- label : Yoyo0901
8- permalink : /yoyo0901
9- description : Yoyo0901
10-
11- cstrikeasia :
12- label : Miyashooooo
13- permalink : /cstrikeasia
14- description : Miyashooooo
15-
16- timyaya8732 :
17- label : Timyaya8732
18- permalink : /timyaya8732
19- description : Timyaya8732
20-
21- ios :
22- label : iOS
23- permalink : /ios
24- description : iOS
25-
26- android :
27- label : Android
28- permalink : /android
29- description : Android
30-
31- swift :
32- label : Swift
33- permalink : /swift
34- description : Swift
35-
36- flutter :
37- label : Flutter
38- permalink : /flutter
39- description : Flutter
40-
41- sandboxie :
42- label : Sandboxie
43- permalink : /sandboxie
44- description : sandboxie
45-
46- tremv :
47- label : TREMV
48- permalink : /tremv
49- description : tremv
6+ rca :
7+ label : RCA
8+ permalink : /rca
9+ description : RCA
Original file line number Diff line number Diff line change 1+ import { JSX } from "react" ;
2+ import styles from "./styles.module.css" ;
3+
4+ type SeverityLevel = "critical" | "high" | "medium" | "low" | "info" ;
5+
6+ type SeverityBadgeProps = {
7+ level : SeverityLevel ;
8+ title ?: string ;
9+ children ?: string ;
10+ }
11+
12+ const severityLabels : Record < SeverityLevel , string > = {
13+ critical : "嚴重" ,
14+ high : "高" ,
15+ medium : "中" ,
16+ low : "低" ,
17+ info : "資訊"
18+ } ;
19+
20+ export default function SeverityBadge ( {
21+ level,
22+ title = "嚴重程度" ,
23+ children
24+ } : SeverityBadgeProps ) : JSX . Element {
25+ const label = children || severityLabels [ level ] ;
26+
27+ return (
28+ < span className = { `${ styles . badge } ${ styles [ level ] } ` } >
29+ < span className = { styles . indicator } />
30+ < span className = { styles . title } > { title } </ span >
31+ < span className = { styles . divider } />
32+ < span className = { styles . label } > { label } </ span >
33+ </ span >
34+ ) ;
35+ }
You can’t perform that action at this time.
0 commit comments