2019年6月10日 星期一

日幣匯率 ( Node-Red 爬蟲 )

日幣匯率 ( Node-Red 爬蟲 )


參考 https://tutorials.webduino.io/zh-tw/docs/socket/useful/exchange-node-red.html


解析匯率網頁

因為要抓取幣值匯率,第一個想到的就是台灣銀行的牌告匯率 ( http://rate.bot.com.tw/xrt?Lang=zh-TW )。


https://rate.bot.com.tw/xrt?Lang=zh-TW

接著按右鍵打開網頁原始碼,看一下我們要的匯率在哪裡,例如我想要抓取日幣現金匯率,那麼就要看到日幣的這個欄位。



接著我們把這段 class 複製下來,這是待會解析網頁時需要抓取的名稱。



搜尋  rate-content-cash.text-right.print_hide

透過 http request 節點來抓取 Opendata,這邊也是一樣的做法,透過 http request 抓取整個牌告匯率網頁,然後透過 html 節點篩選出特定內容,由 function 節點做分析,最後將結果由 debug 顯示出來。




inject node設定
http request 設定
https://rate.bot.com.tw/xrt?Lang=zh-TW

  點選 http request 打開設定,於 URL 的地方填入牌告匯率網址:https://rate.bot.com.tw/xrt?Lang=zh-TW 





再來點選 html 打開設定,Select 的地方填入 .rate-content-cash.text-right.print_hide ,注意這邊的 Select 是 CSS 選擇器,就跟我們在寫 CSS 是一樣,例如我們要抓取的 class 在 HTML 呈現是 class=”rate-content-cash text-right print_hide”,選擇器就變成 .rate-content-cash.text-right.print_hide
Output 選擇 the html content 或 only the text content 都可以,設定好之後可以賦予這個標籤一個名稱,我這邊給它命名為 Select_Filter。
這時候我們先不要急著設定 function 內容,先把這個流程 Deploy,點選 inject 看一下結果會是怎樣,結果呈現一組陣列,這組陣列表示整份網頁裡面 .rate-content-cash.text-right.print_hide 的值依序排列,而我們要的日幣匯率就在劃線的地方,這邊可能就要手動數一下,日幣匯率是排在陣列的第幾個位置,這邊我算出來是在第 15 與 16 位,對應到陣列就是 14 和 15 ( 陣列第一個位置為 0 開始算 ),而第一個日幣匯率是銀行買入的不用看,要買日幣就要看第二個銀行賣出,也就是第 15 位。



接著點選 function 打開設定,輸入下面這段程式碼,最後顯示的結果就會是「( 時:分:秒 ) 日幣匯率:....」。




var jp = msg.payload[15];
var date = new Date();
var h = date.getHours();
var m = date.getMinutes();
var s = date.getSeconds();
if(h<10){
    h = '0'+h;
}
if(m<10){
    m = '0' + m;
}
if(s<10){
    s = '0' + s;
}
msg.payload = '(' + h + ':' + m + ':' + s + ')\n'+
'日幣匯率:' + jp ;

msg.rate = jp;
if(jp<0.30){
    msg.jp29 = 'on' ;
}else{
    msg.jp29 = 'off' ;
}

return msg;

到達匯率時,寄送 Email 並透過 ESP8266 MQTT點亮燈泡
接著點選 function 打開設定,輸入下面這段程式碼

var jp = msg.payload[15];
msg.rate = jp;
if(jp<0.30){
    msg.payload = 'on' ;
}else{
    msg.payload = 'off' ;
}
return msg;

設定 匯率 < 0.30 ON 否則 OFF  (0.30 或是其他數值 控制On/Off)






 設定MQTT server 配合ESP8266 NodeMCU u'P 控制LED ON /LED OFF


(JPY) 這個 function 節點,這邊就純粹針對匯率做判斷,如果小於多少就傳出 on,不然就傳出 off。   
if(jp<0.30){
    msg.jp29 = 'on' ;
}else{
    msg.jp29 = 'off' ;
var jp = msg.payload[15];
var date = new Date();
var h = date.getHours();
var m = date.getMinutes();
var s = date.getSeconds();
if(h<10){
    h = '0'+h;
}
if(m<10){
    m = '0' + m;
}
if(s<10){
    s = '0' + s;
}
msg.payload = '(' + h + ':' + m + ':' + s + ')\n'+
'日幣匯率:' + jp ;

msg.rate = jp;
if(jp<0.30){
    msg.jp29 = 'on' ;
}else{
    msg.jp29 = 'off' ;
}

return msg;


比較特別的是用到了一個 switch 的流程,目的在於判斷如果是 on 就走寄信的流程,如果是 off 則顯示訊息就好。






接著 set mail  這個 function 節點,


var jp = msg.rate;
var date = new Date();
var h = date.getHours();
var m = date.getMinutes();
var s = date.getSeconds();
if(h<10){
    h = '0'+h;
}
if(m<10){
    m = '0' + m;
}
if(s<10){
    s = '0' + s;
}
msg.to = 'alex.pts2594@gmail.com';
msg.topic = '該買日幣囉!('+jp+')';
msg.payload = '(' + h + ':' + m + ':' + s + ') '+
'日幣匯率:' + jp ;
return msg;
接著 send gmail  這個 email 節點,





Depoly 之後點選 inject 啟動,可以設定每一分鐘或每兩分鐘抓取一次匯率,當匯率到達一定數值的時候,就會自動寄發 Email 並且點亮LED燈泡囉!




另外利用MQTT in 檢視送出的資料是否正確或是ESP8266 NodeMcu 回送的資料是否正確
Node-RED 送出on , off 可以利用ESP8266 NodeMcu 程式 回傳A , B 或是0 ,1 信號





設定MQTT IN
再來設定顯示畫面








[{"id":"75d62350.b6310c","type":"mqtt out","z":"17bb63ed.68b25c","name":"JP Rate","topic":"alex9ufo/Jprate","qos":"0","retain":"true","broker":"b0f7c1ff.aede5","x":700,"y":340,"wires":[]},{"id":"dd75c89f.246658","type":"inject","z":"17bb63ed.68b25c","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":280,"wires":[["b984576b.3db788"]]},{"id":"b984576b.3db788","type":"http request","z":"17bb63ed.68b25c","name":"","method":"GET","ret":"txt","url":"https://rate.bot.com.tw/xrt?Lang=zh-TW","tls":"","x":250,"y":280,"wires":[["a586a73d.e7e968"]]},{"id":"742c5861.77e038","type":"function","z":"17bb63ed.68b25c","name":"(JPY) ","func":"var jp = msg.payload[15];\nvar date = new Date();\nvar h = date.getHours();\nvar m = date.getMinutes();\nvar s = date.getSeconds();\nif(h<10){\n    h = '0'+h;\n}\nif(m<10){\n    m = '0' + m;\n}\nif(s<10){\n    s = '0' + s;\n}\nmsg.payload = '(' + h + ':' + m + ':' + s + ')\\n'+\n'日幣匯率:' + jp ;\n\nmsg.rate = jp;\nif(jp<0.30){\n    msg.jp29 = 'on' ;\n}else{\n    msg.jp29 = 'off' ;\n}\n\nreturn msg;\n","outputs":1,"noerr":0,"x":190,"y":440,"wires":[["2647bcb6.04b4f4","88d5c709.ef2d48"]]},{"id":"85a49c7e.8c917","type":"e-mail","z":"17bb63ed.68b25c","server":"smtp.gmail.com","port":"465","secure":true,"name":"alex.pts2594@gmail.com","dname":"send gmail","x":690,"y":400,"wires":[]},{"id":"4fe101eb.f32a6","type":"function","z":"17bb63ed.68b25c","name":"set mail","func":"var jp = msg.rate;\nvar date = new Date();\nvar h = date.getHours();\nvar m = date.getMinutes();\nvar s = date.getSeconds();\nif(h<10){\n    h = '0'+h;\n}\nif(m<10){\n    m = '0' + m;\n}\nif(s<10){\n    s = '0' + s;\n}\nmsg.to = 'alex.pts2594@gmail.com';\nmsg.topic = '該買日幣囉!('+jp+')';\nmsg.payload = '(' + h + ':' + m + ':' + s + ') '+\n'日幣匯率:' + jp ;\nreturn msg;","outputs":1,"noerr":0,"x":540,"y":420,"wires":[["85a49c7e.8c917","5883fca3.223114"]]},{"id":"bd11d496.74e128","type":"debug","z":"17bb63ed.68b25c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":690,"y":480,"wires":[]},{"id":"88d5c709.ef2d48","type":"switch","z":"17bb63ed.68b25c","name":"","property":"jp29","propertyType":"msg","rules":[{"t":"eq","v":"on","vt":"str"},{"t":"neq","v":"on","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":370,"y":440,"wires":[["4fe101eb.f32a6"],["bd11d496.74e128"]]},{"id":"a586a73d.e7e968","type":"html","z":"17bb63ed.68b25c","name":"Select_Filter","property":"payload","tag":".rate-content-cash.text-right.print_hide","ret":"html","as":"single","x":210,"y":360,"wires":[["742c5861.77e038","1d4ce786.7df658","60b8b7b5.42b3a8"]]},{"id":"990be0ca.6ac98","type":"mqtt in","z":"17bb63ed.68b25c","name":"Jp Rate","topic":"alex9ufo/Jprate","qos":"2","broker":"b0f7c1ff.aede5","x":70,"y":540,"wires":[["802395c2.98fff8","af2c3d09.0421c","fc81fb8.f3b5108"]]},{"id":"af2c3d09.0421c","type":"debug","z":"17bb63ed.68b25c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":270,"y":580,"wires":[]},{"id":"802395c2.98fff8","type":"ui_switch","z":"17bb63ed.68b25c","name":"","label":"switch","group":"68a79aa3.bcb534","order":0,"width":0,"height":0,"passthru":true,"decouple":"false","topic":"","style":"","onvalue":"on","onvalueType":"str","onicon":"","oncolor":"","offvalue":"off","offvalueType":"str","officon":"","offcolor":"","x":250,"y":540,"wires":[[]]},{"id":"5883fca3.223114","type":"debug","z":"17bb63ed.68b25c","name":"","active":false,"tosidebar":false,"console":false,"tostatus":false,"complete":"payload","x":690,"y":440,"wires":[]},{"id":"1d4ce786.7df658","type":"debug","z":"17bb63ed.68b25c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":490,"y":280,"wires":[]},{"id":"2647bcb6.04b4f4","type":"debug","z":"17bb63ed.68b25c","name":"","active":true,"tosidebar":false,"console":false,"tostatus":false,"complete":"jp29","x":380,"y":500,"wires":[]},{"id":"60b8b7b5.42b3a8","type":"function","z":"17bb63ed.68b25c","name":"(JPY) on off","func":"var jp = msg.payload[15];\nmsg.rate = jp;\nif(jp<0.29){\n    msg.payload = 'on' ;\n}else{\n    msg.payload = 'off' ;\n}\nreturn msg;","outputs":1,"noerr":0,"x":490,"y":340,"wires":[["75d62350.b6310c"]]},{"id":"fc81fb8.f3b5108","type":"ui_text","z":"17bb63ed.68b25c","group":"68a79aa3.bcb534","order":0,"width":0,"height":0,"name":"","label":"text","format":"{{msg.payload}}","layout":"row-spread","x":250,"y":620,"wires":[]},{"id":"b0f7c1ff.aede5","type":"mqtt-broker","z":"","name":"mqtt","broker":"broker.mqtt-dashboard.com","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"true","birthPayload":""},{"id":"68a79aa3.bcb534","type":"ui_group","z":"","name":"alexJp_Rate","tab":"a0a49c3.906956","order":1,"disp":true,"width":"6","collapse":false},{"id":"a0a49c3.906956","type":"ui_tab","z":"","name":"alex9ufo JP Rate","icon":"dashboard"}]


沒有留言:

張貼留言

Messaging API作為替代方案

  LINE超好用功能要沒了!LINE Notify明年3月底終止服務,有什麼替代方案? LINE Notify將於2025年3月31日結束服務,官方建議改用Messaging API作為替代方案。 //CHANNEL_ACCESS_TOKEN = 'Messaging ...