Treffer: 巨量資料處理系統之分析與比較 ; The analysis of big data processing systems

本研究蒐集國內外巨量資料相關之技術、系統議題相關的文獻來深入探討、比較巨量資料處理技術、系統間的優勢、劣勢，為了提供使用者更良好的巨量資料系統操作環境，本研究提出一個能半自動化架設系統並且幫助使用者解決系統問題的巨量資料處理系統，架構分成兩部分主要的功能：1.巨量資料系統架設 2.系統問題解決方案。 在系統使用介面上，透過寫好的巨量資料架設腳本能幫助使用者快速架設Spark、Hadoop等巨量資料系統，減少使用者在系統架設上花費的時間成本；若使用者遇到系統相關的問題，透過本系統問題解決方案的功能，蒐集了部分使用相關的解決方案，能改善Hadoop技術在使用時常遇到的NameNode、DataNode not running的問題，另外也能解決ssh免密碼登入失敗、No such file or directory、command not found的問題。希望透過本研究延伸的發展方向，在這套系統上開發更多功能、問題解決方案，對未來處理巨量資料系統發展有所幫助，提供未來相關研究作建議、參考。 ; This study first reviewed literature on big data technology and systems to compare the strengths and weaknesses of various systems. To provide users with a more efficient operating environment for big data systems, this study introduced a big data processing system capable of setting up systems semi-automatically as well as helping users solve system problems. The proposed system includes two major functions: (a) set up big data systems, and (b) provide solutions to system problems. The designed instructions can effectively assist users set up big data systems such as Spark and Hadoop; and thus the time involved with system set up for users is decreased When users experience system-related problems, such as “NameNode not running” , “DataNode not running”, “ssh password-free log in failure”, “no such file or directory,” and “command not found.”, users may employ the problem solution function to solve problems commonly encountered. For future study, more functions and problem solutions can be developed for the system. Such improvements could be beneficial to future big data system development as well as recommendations to related studies. ; 碩士