{"ops":[{"insert":"了解 Whistle 的工作原理以及如何實現一個簡單的數據包捕獲調試工具。\n\nGithub地址：https://github.com/avwo/whistle\n\nWhistle 是一個基於 Node.js 的跨平台 Web 數據包捕獲和調試 (HTTP) 代理。\n\n實時抓包：支持HTTP、HTTPS、HTTP2、WebSocket、TCP等常見Web請求的抓包。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"修改請求響應：與常見的抓包調試工具使用的斷點不同，Whistle 使用類似於系統主機的配置規則方法。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴展：支持通過Node編寫插件，或者作為獨立的npm包引入項目兩種擴展方式。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/2a86d79e266160ac36149b9d5d3c5cfcde827245.jpg","width":1080,"height":785,"size":288362,"status":"loaded"}}},{"insert":"\n本文從基礎開始逐步引導您完成 Whistle 功能：\n\n什麼是 HTTP 代理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"實現一個簡單的 HTTP 代理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"完整的 HTTP 代理架構（Whistle）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"具體實現原理"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"資源"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n1.什麼是HTTP代理"},{"attributes":{"header":1},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/ac15448058ace757f5714cc54d546821b09c1dae.jpg","width":999,"height":460,"size":71683,"status":"loaded"}}},{"insert":"\n代理是客戶端到服務器的傳輸服務，其中：\n\n免代理：客戶端與服務器建立直連後，即可進行數據交換。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"代理請求：客戶端不直接與服務器建立連接。相反，客戶端與代理建立連接後，將目標服務器的地址發送給代理，然後通過代理與服務器建立連接。如果代理服務是 HTTP Server，則稱為 HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n讓我們看看客戶端如何將目標服務器地址傳遞給 HTTP 代理，以及 HTTP 代理如何建立與目標服務器的連接。\n\n2.實現簡單的HTTP代理"},{"attributes":{"header":1},"insert":"\n"},{"insert":"讓我們從 Node.js 中實現的最簡單的 HTTP 代理開始：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/b56aa85d4fbf7b0a6d259f658d16f1474ed78e52.jpg","width":1820,"height":1192,"size":673936,"status":"loaded"}}},{"insert":"\n上述代碼實現了一個具有轉發請求功能的HTTP代理。從代碼中可以看出，HTTP代理就是一個普通的HTTP Server，監聽request和connect兩個事件。客戶端會通過這兩個事件傳遞目標服務器的地址。\n\n"},{"attributes":{"color":"#ff6699"},"insert":"request"},{"insert":"：普通HTTP通過這個事件傳遞目標服務器的地址。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"attributes":{"color":"#ff6699"},"insert":"connect"},{"insert":"：非HTTP請求，如HTTPS、HTTP/2、WebSocket、TCP等，使用該事件發送目標服務器的地址。觸發此事件的代理請求也稱為隧道代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n請求。url 或 req.headers。主機可以在事件中獲取目標服務器的地址（host:port），與服務器的地址建立連接，並將結果以HTTP響應的方式回傳給客戶端。除了請求轉發，完整的 HTTP 至少應該有：\n\n查看實時抓包；"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"解析 HTTPS 請求；"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"修改請求響應內容。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴展功能。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n以 Whistle 為例，我們將了解如何使用 Node.js 實現一個完整的 HTTP 代理。\n\n3.完整的HTTP代理架構（Whistle）"},{"attributes":{"header":1},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/60320844464b57e00b394ddc44cc5eaee26cdfa7.jpg","width":1080,"height":543,"size":203256,"status":"loaded"}}},{"insert":"\n主要分為五個模塊：\n\n請求訪問模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"隧道代理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"處理HTTP請求模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"規則管理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"插件管理模塊"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n4.具體實施原則"},{"attributes":{"header":1},"insert":"\n"},{"insert":"下面我們來看看這五個模塊是如何實現的。\n\n4.1 請求訪問模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/bb4630d53ad3ac90700c3e75b71baa6cb592a86a.jpg","width":1080,"height":496,"size":70978,"status":"loaded"}}},{"insert":"\n所有請求首先通過請求訪問模塊。Whistle 支持四種請求訪問模式：\n\nHTTP HTTPS 直接請求：通過配置主機或 DNS 將請求轉發給 Whistle。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"HTTP 代理：默認 Whistle 訪問方式通過瀏覽器插件配置系統代理或 HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"HTTPS 代理：對 HTTP 代理（即 HTTPS Server）上的代理請求進行加密。您可以使用指定的證書將請求傳輸到 HTTP 代理。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Socks5 代理：使用 NPM 包"},{"attributes":{"color":"#ff6699"},"insert":"socksv5"},{"insert":"將 TCP 請求轉換為普通的 TCP 請求，將 TCP 請求轉換為隧道代理請求。"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n基本實現原理是：將所有請求轉換為TUNNEL代理請求或HTTP代理的HTTP請求，再將隧道代理請求解析為HTTP請求。\n\n如何將普通的TCP請求轉化為隧道代理請求？有關詳細信息，請參閱黑色代理\n\n讓我們看看如何從隧道代理請求中解析 HTTP 請求。\n\n4.2 隧道代理模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/e03667f297439e613edb6d1585cd329f05eb50cc.jpg","width":1080,"height":375,"size":72962,"status":"loaded"}}},{"insert":"\n關鍵點（HTTP請求也可以通過隧道代理）：\n\n1. 匹配的全局規則決定是否解析隧道代理請求。如果否，則隧道代理請求被視為普通 TCP 請求。\n\n2. Socket.once('data', handler) 必要時讀取請求點的第一幀；\n\n3. 將第一幀數據轉換成字符串，通過正則/^(\\w+)\\s+(\\s+)HTTP\\\\s+/1\\d$/mi是HTTP請求嗎？如果是HTTP請求，檢查是否是CONNECT請求，即TUNNEL代理請求（隧道代理請求也可以是隧道代理請求）。如果是，則將請求轉發給隧道代理方法進行處理。如果否，則將請求轉發給HTTP請求模塊進行處理。\n\n4. 如果不是 HTTP 請求，則視為 HTTPS 請求。在這種情況下，使用中間人將 HTTPS 請求轉換為 HTTP 請求。\n\n5. Whistle 首先按以下順序獲取請求的證書：\n\n通過匹配插件（可以通過規則"},{"attributes":{"color":"#ff6699"},"insert":"sniCallback://plugin"},{"insert":"指定插件加載證書）；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"通過引導參數"},{"attributes":{"color":"#ff6699"},"insert":"-z certDir"},{"insert":"指定目錄或"},{"attributes":{"color":"#ff6699"},"insert":"~/.WhistleAppData/custom_certs"},{"insert":"加載自定義證書；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"如果您沒有這些自動證書，Whistle 會自動生成一個默認證書。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n6. 獲取證書後，使用該證書啟動HTTPS Server，將HTTPS請求轉換為HTTP請求，提交給HTTP請求模塊處理。\n\n4.3 HTTP請求處理模塊"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/02cfb1bfb696e704a3cba1037c571b576aebc0b4.jpg","width":1002,"height":310,"size":75317,"status":"loaded"}}},{"insert":"\nHTTP請求處理可以分為兩個階段：\n\n1. 請求階段：\n\n匹配全局規則；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"如果有類似口哨的規則。http://whistle.xxx，執行對應的插件鉤子，檢索插件規則，並與匹配的全局規則合併。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"執行規則，記錄狀態，並向指定服務發出請求。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. 反應階段：\n\n執行匹配插件鉤子獲取插件規則並將其與匹配的全局規則合併。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"執行規則，記錄狀態，並請求返回給客戶端。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n4.4 規則管理"},{"attributes":{"header":2},"insert":"\n"},{"insert":"與使用斷點修改請求響應數據的傳統數據包捕獲調試代理不同，Whistle 使用配置規則修改請求響應數據。這種方法的優點是操作簡單、持久存儲和操作共享。讓我們看一些例子：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/3f911b0eaebbabbdc14eac008c28ddeb30fc9931.jpg","width":1080,"height":278,"size":136381,"status":"loaded"}}},{"insert":"\n"},{"attributes":{"bold":true},"insert":"Whistle 的規則管理有兩個主要功能："},{"insert":"\n\n解析規則"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"匹配規則"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n解析規則"},{"attributes":{"header":1},"insert":"\n"},{"attributes":{"bold":true},"insert":"Whistle 有兩種規則："},{"insert":"\n\n1. 全局規則（通用規則），所有請求都會嘗試匹配的規則，由以下規則組成：\n\n規則 表示配置的規則。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n插件根目錄rules.txt配置文件；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n 文件：https://github.com/whistle-plugins/whistle.autosave/blob/master/rules.txt\n\n接口或插件rules.txt通過@URL引入的遠程規則（單行，Whistle定期更新遠程規則）。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/d46e9a1be0c1d85642d660455a87e259b2eacd6a.jpg","width":708,"height":496,"size":125190,"status":"loaded"}}},{"insert":"\n2. 插件規則（私有規則）是只有在全局規則中匹配 whistle.xxx 協議的插件請求才會匹配的規則。它們由以下規則組成：\n\nreqRulesServer 等插件動態返回鉤子；"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n插件根目錄_rules.txt中配置的靜態規則。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":" 文件：https://wproxy.org/whistle/plugins.html\n\n匹配規則"},{"attributes":{"header":1},"insert":"\n"},{"insert":"Whistle 規則的完整結構是：\n\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/f8d4db5451b3993765d04b4e89376f6a5948d021.jpg","width":998,"height":402,"size":106309,"status":"loaded"}}},{"insert":"\n文件：https://wproxy.org/whistle/mode.html\n"},{"attributes":{"header":2},"insert":"\n"},{"insert":"4.5 插件管理"},{"attributes":{"header":2},"insert":"\n"},{"insert":"\n"},{"attributes":{"class":"normal-img"},"insert":{"native-image":{"alt":"read-normal-img","url":"https://b1.sanwen.net/b_article/bcb6c5ffd9e5ecb1d0984da15e02fae19b9b29e5.jpg","width":1080,"height":194,"size":87264,"status":"loaded"}}},{"insert":"\nWhistle 插件有很多功能。它不僅擁有 Node 的所有功能，還可以運行 Whistle 的所有規則。它用於執行以下操作：\n\n認證功能"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"提供UI界面"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"充當請求服務器（直接響應或轉發和修改請求響應）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"統計請求信息（查看報告/日誌數據等）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"設置規則（動態、靜態、全局和私有規則）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"獲取抓包數據"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"編解碼器請求響應數據流（管道流功能）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"擴展界面右鍵菜單（如分享抓包數據）"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"保存和同步規則值數據"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"自定義 HTTPS 請求證書"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n如：\n\nWhistle. script：通過自定義腳本實現規則的動態設置"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. vase：提供靈活強大的模擬能力"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. inspect：輕鬆快速注入vConsole、Eruda等頁面調試工具"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"Whistle. sni-callback：自定義證書插件"},{"attributes":{"list":"ordered"},"insert":"\n"},{"insert":"\n其他插件示例見：https://github.com/whistle-plugins\n\nWhistle 如何作為插件發揮作用？遵循以下三個設計原則：\n\n1. "},{"attributes":{"bold":true},"insert":"完整性"},{"insert":"：\n\n確保所有功能點都是可擴展的，例如請求認證、證書生成、抓包、規則設置和請求處理。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. "},{"attributes":{"bold":true},"insert":"穩定"},{"insert":"：\n\n內部異常不會影響其他功能。每個 Whistle 插件都是一個獨立的進程，插件通過 HTTP 與 Whistle 交互。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\nWhistle 使用 NPM 包 "},{"attributes":{"color":"#ff6699"},"insert":"pfork"},{"insert":" 來啟動插件進程。進程間的切換直接通過Node的HTTP模塊實現。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n3. "},{"attributes":{"bold":true},"insert":"使用方便"},{"insert":"：\n\n方便用戶開發和使用。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n4. "},{"attributes":{"bold":true},"insert":"開發"},{"insert":"：結構簡單（NPM包）+腳手架"},{"attributes":{"color":"#ff6699"},"insert":"lack"},{"insert":"\n\n使用： 安裝npm包，使用同內置協議，內置交互界面。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n有關插件的更多詳細信息，請參閱：https://wproxy.org/whistle/plugins.html\n\n事實上，Whistle 支持插件擴展，也可以在項目中作為獨立模塊使用；除了本地開發外，Whistle 還可用於開發多人開發和協調工具。比如Whistle的實現原理，後面會介紹。\n\n1. Whistle基於多人多環境遠程抓包調試工具的實現\n\nNohost：https://github.com/Tencent/nohost"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n2. TDE是一款基於Whistle和Nohost的分佈式遠程抓包調試工具\n\nTDE目前僅在騰訊使用，將逐步開源。"},{"attributes":{"list":"bullet"},"insert":"\n"},{"insert":"\n5. 參考資料"},{"attributes":{"header":1},"insert":"\n"},{"insert":"Github 存儲庫：https://github.com/avwo/whistle\n官方插件庫：https://github.com/whistle-plugins\n詳細文檔：https://wproxy.org/whistle/\n\n網站地址：https://www.mo4tech.com/implementation-principle-of-whistle.html\n翻譯：阿曜ちゃん\n\n最後，謝謝大家的瀏覽，十分感謝大家的支持。\n"}]}
标签：
