tag:blogger.com,1999:blog-70577340951025777532024-02-07T21:47:27.702-08:00筆記本Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.comBlogger32125tag:blogger.com,1999:blog-7057734095102577753.post-38794885073915639362017-10-11T22:17:00.001-07:002017-10-11T22:17:43.399-07:00機器學習:支持智能工廠決策的決定<div class="article-top-bar solid-bg" style="background-color: white; font-family: Raleway, Helvetica, Arial, Verdana, sans-serif; font-size: 18px; margin: -40px 0px 44px; overflow: hidden; padding: 0px; position: relative; z-index: 4;">
<div class="wf-wrap" style="box-sizing: border-box; margin: 0px auto; max-width: 100%; padding: 0px 20px; width: 1250px;">
<div class="wf-container-top" style="border-bottom: 1px solid rgba(0, 0, 0, 0.15); border-left-color: rgba(0, 0, 0, 0.15); border-right-color: rgba(0, 0, 0, 0.15); border-top-color: rgba(0, 0, 0, 0.15); display: table; margin: 0px; padding: 0px 0px 10px; width: 1170px;">
<div class="entry-meta" style="display: table-cell; float: left; font-size: 16px; line-height: 25px; margin: 0px; overflow: hidden; padding: 0px; vertical-align: middle;">
<br />
Source: http://www.produzioneperfetta.it/machine-learning-previsioni-supporto-delle-decisioni-nelle-smart-factory/<br />
<br />
<time class="entry-date updated" datetime="2016-06-23T17:25:38+00:00" style="background-position: 0px center; background-repeat: no-repeat; color: rgba(0, 0, 0, 0.5); display: inline-block; margin: 0px; padding: 0px; position: relative; transition: all 0.05s linear 0.05s;">2016年6月23日</time><span class="category-link" style="background-position: 0px center; background-repeat: no-repeat; color: rgba(0 , 0 , 0 , 0.5); display: inline-block; margin: 0px 20px 0px 0px; padding: 0px 0px 0px 25px; position: relative;"><a href="http://www.produzioneperfetta.it/category/smart-factory/" style="color: rgba(0, 0, 0, 0.5); margin: 0px; padding: 0px; text-decoration-line: none; transition: all 0.05s linear 0.05s;">智能工廠</a></span><a class="author vcard" href="http://www.produzioneperfetta.it/author/alessio/" rel="author" style="background-image: url("data:image/svg+xml,%3Csvg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' x='0px' y='0px' width='16px' height='16px' viewBox='0 0 16 16' enable-background='new 0 0 16 16' xml:space='preserve'%3E%3Cpath fill='rgba(0,0,0,0.5)' d='M14.429,12.209c-0.176-0.526-0.496-1.06-0.965-1.372c-0.908-0.603-1.964-0.923-2.985-1.266c-0.247-0.086-0.496-0.18-0.722-0.314c-0.202-0.12-0.275-0.364-0.323-0.579c-0.021-0.112-0.032-0.231-0.04-0.346c0.74-1.014,1.231-2.666,1.231-4.021c0-2.117-1.195-2.707-2.668-2.707c-1.476,0-2.671,0.589-2.671,2.705c0,1.404,0.527,3.127,1.311,4.128C6.59,8.515,6.582,8.596,6.568,8.673C6.521,8.888,6.446,9.129,6.244,9.248c-0.226,0.133-0.475,0.22-0.722,0.304C4.5,9.896,3.445,10.199,2.539,10.803c-0.471,0.309-0.79,0.88-0.967,1.406c-0.181,0.551-0.255,1.229-0.244,1.78H8h6.672C14.685,13.438,14.61,12.76,14.429,12.209z'/%3E%3C/svg%3E"); background-position: 0px center; background-repeat: no-repeat; color: rgba(0, 0, 0, 0.5); display: inline-block; margin: 0px 20px 0px 0px; padding: 0px 0px 0px 25px; position: relative; text-decoration-line: none; transition: all 0.05s linear 0.05s;" title="查看Alessio Passalacqua發表的所有帖子"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">由</span><span class="fn" style="margin: 0px; padding: 0px;">Alessio Passalacqua</span></a></div>
<div class="navigation-inner" style="display: table-cell; margin: 0px 0px 5px; min-width: 88px; overflow: hidden; padding: 0px; vertical-align: middle;">
<div class="single-navigation-wrap" style="float: right; margin: 0px -6px 0px 0px; padding: 0px;">
<a class="prev-post" href="http://www.produzioneperfetta.it/data-driven-decision/" rel="next" style="background-image: url("data:image/svg+xml,%3Csvg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' x='0px' y='0px' width='12px' height='12px' viewBox='0 0 12 12' enable-background='new 0 0 12 12' xml:space='preserve'%3E%3Cpolygon fill='rgba(0,0,0,0.5)' points='8.088,9.529 4.551,5.997 8.078,2.47 6.664,1.055 1.713,6.005 3.128,7.42 3.134,7.414 6.672,10.946 '/%3E%3C/svg%3E"); background-position: center center; background-repeat: no-repeat; color: #4d71da; float: left; height: 12px; margin: 0px; padding: 5px 4px 5px 6px; text-decoration-line: none; width: 12px;"></a><a class="next-post" href="http://www.produzioneperfetta.it/smart-factory-tra-dire-e-fare/" rel="prev" style="background-image: url("data:image/svg+xml,%3Csvg version='1.1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' x='0px' y='0px' width='12px' height='12px' viewBox='0 0 12 12' enable-background='new 0 0 12 12' xml:space='preserve'%3E%3Cpolygon fill='rgba(0,0,0,0.5)' points='10.286,6.006 10.279,5.999 10.279,5.999 8.865,4.583 8.864,4.584 5.335,1.055 3.921,2.47 7.449,5.998 3.913,9.529 5.326,10.943 8.863,7.412 8.871,7.42 '/%3E%3C/svg%3E"); background-position: center center; background-repeat: no-repeat; color: #4d71da; float: left; height: 12px; margin: 0px; padding: 5px 6px; text-decoration-line: none; width: 12px;"></a></div>
</div>
</div>
</div>
</div>
<div class="content" id="content" role="main" style="background-color: white; box-sizing: border-box; float: left; font-family: Raleway, Helvetica, Arial, Verdana, sans-serif; font-size: 18px; margin: 0px; padding: 0px 20px; width: 895.391px;">
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigDWXBpHpiS5uh4kHfCNb9RVkHkBkbbFwSiTAiInZV_CQfqOyQG5pcBF1VxfAQ8N8jsQ-l6_JhTi_aVRU1ZxV6oF8TcUn90xs9_3Wwm-bcvWA_lJPsMIK4B3lr1AEQ79urTfUH4oyfsI3B/s1600/smart+factory.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="520" data-original-width="783" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigDWXBpHpiS5uh4kHfCNb9RVkHkBkbbFwSiTAiInZV_CQfqOyQG5pcBF1VxfAQ8N8jsQ-l6_JhTi_aVRU1ZxV6oF8TcUn90xs9_3Wwm-bcvWA_lJPsMIK4B3lr1AEQ79urTfUH4oyfsI3B/s640/smart+factory.png" width="640" /></a></div>
<article class="post-461 post type-post status-publish format-standard has-post-thumbnail category-smart-factory tag-data-driven-innovation tag-industry-4-0 tag-smart-factory description-off" id="post-461" style="margin: 0px; padding: 0px; position: relative;"><span class="ink" style="background: rgba(255, 255, 255, 0.38); border-radius: 50%; display: block; height: 855px; margin: 0px; padding: 0px; position: absolute; transform: scale(0); width: 855px;"></span><br /><i style="background: linear-gradient(30deg, rgba(53, 95, 190, 0.3) 0%, rgba(53, 196, 244, 0.3) 100%); height: 531px; left: 0px; margin: 0px; opacity: 0; padding: 0px; position: absolute; top: 0px; transition: all 450ms ease; width: 855.391px;"></i><div style="margin-bottom: 10px; padding: 0px;">
<em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;"><br /></span></em>
<em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;"><br /></span></em>
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">智能工廠</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這個術語</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">是指促進信息的實時傳輸和共享的公司模式,目的是創建專注於</span></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">數據驅動的</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">決策策略</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(包括運營和戰略)的</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">智能生產</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">,以創建理想的條件優化和運營效率。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">由於融合和整合到工業技術系統中,世界正處於創新和變革的新時代的門檻,雖然不是全新的,但現在可以通過低成本加上功率增加。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數字革命,</span></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">工業4.0</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">,是建立在它們之間創造協同效應以最大限度發揮潛力的能力。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">我們談論低成本和新的連接水平,包括</span></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">物聯網</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">,物聯網,高級數據</span><span style="font-weight: 700; margin: 0px; padding: 0px;">分析</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">和</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">大數據</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">,</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">機器學習</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">和</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">預測分析</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">技術</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">在</span><span style="font-weight: 700; margin: 0px; padding: 0px;">數字化革命</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">創造了公司獲取數據,並將它們轉化成信息,目的是減少通過流程自動化和優化的效率成本的價值。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據本身沒有值。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據可以提供知識,只有根據</span></span><a href="http://www.produzioneperfetta.it/data-driven-innovation/" style="color: #4d71da; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據驅動的創新過程</span></a><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">進行分析,才可以回答具體業務的關鍵問題</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據無聲,數據分析提供了一種通用的語言。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據驅動的支持流程從總結關鍵業務問題的問題開始。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">第二個關鍵步驟是弄清楚你所擁有的數據是否足夠回答這個問題,或者獲得最好的方法是什麼。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">因此,開始</span></span><a href="http://www.produzioneperfetta.it/analisi-dati-istruzioni-per-luso-manuale-utente/" style="color: #4d71da; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據分析</span></a><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">的最方便的方面</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據分析的核心是</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">機器學習</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">方法</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">但“ </span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">機器學習</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;"> ” </span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">的方法是什麼</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">?</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這些是允許您從數據中提取信息的算法類型。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">它們可以用於非常具體的任務,而不需要明確的編程。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">通過克服傳統編程的局限性,您可以編寫程序來直接解決特定問題。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">重要的是要了解機器學習方法沒有要求替代人類智力,而是通過將原始數據轉化為新知識來支持決策的功能。</span></div>
<blockquote style="background-color: rgba(136, 136, 136, 0.08); border-radius: 4px; box-sizing: border-box; color: #333333; font-stretch: normal; line-height: 27px; margin: 0px 0px 10px; padding: 20px 25px;">
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">將公司轉化為“智慧工廠”不僅僅是技術問題,也是文化問題。</span></div>
</blockquote>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">實際上,要了解如何有效地應用機器學習方法,你不必理解所使用的技術,但是足夠了解這些方法可以解答什麼問題。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">機器學習方法可以主要解決這四種類型的問題:</span></div>
<ul style="list-style-image: none; list-style-position: outside; margin: 0px 0px 10px 20px; padding: 0px;">
<li style="margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">正在考慮的物業的價值是什麼?</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(我們再談談回歸)</span></span></li>
<li style="margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">給定元素屬於哪個類別或類別?</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(分級)</span></span></li>
<li style="margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這些對像是否相似?</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(聚類)</span></span></li>
<li style="margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這個值很奇怪?</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(異常檢測)</span></span></li>
</ul>
<h2 style="clear: both; color: #333333; font-size: 36px; font-stretch: normal; font-weight: normal; line-height: 42px; margin: 0px 0px 10px; padding: 0px; text-align: center;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">預測維護</span></h2>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">在大型跨國公司進行業務生產的過程中,</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">大數據</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">已成為</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">智能數據</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">:可以智能解決特定生產問題的數據的一部分。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">由於</span></span><em style="margin: 0px; padding: 0px;">數據驅動的</em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">策略</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">,這些公司正在增加市場,並能夠及時採用最佳策略,從競爭對手中脫穎而出。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">數據分析導致洞察; </span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">企業家和經理可以將這些見解轉化為改善不同業務方面的決策和行動。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">我們認為</span><a href="https://www.linkedin.com/pulse/amazing-ways-big-data-drives-success-rolls-royce-bernard-marr?trk=mp-reader-card" style="color: #4d71da; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">勞斯萊斯</span></a><span style="margin: 0px; padding: 0px; vertical-align: inherit;">在1971年失敗後改變了業務,從豪華機器製造商到波音飛機的發動機製造商。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">創新和技術先進的推進發動機的市場,其中故障可能導致重大的經濟損失。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">Rolls Royce通過一個名為“發動機健康管理”的計劃,通過收集位於其上的200多個傳感器產生的數據並通過衛星傳輸來監測世界各地的1,000多台發動機的健康狀況。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">由於精心收集的數據和復雜的分析技術,分析師能夠提前預測和識別可能的故障和問題。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">減少停機,維護的風險和成本。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">那我們來談談</span></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">預測維護</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這種方法對於保持設備運行,最大化使用和性能以及降低計劃外停機時間的成本至關重要。</span></span></div>
<h2 style="clear: both; color: #333333; font-size: 36px; font-stretch: normal; font-weight: normal; line-height: 42px; margin: 0px 0px 10px; padding: 0px; text-align: center;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">虛擬計量與異常檢測</span></h2>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">我們考慮到微處理器行業和復雜的製造工藝,使矽塊成為高性能半導體。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">一個非常複雜的生產過程,由數百個加工操作的序列組成。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">在此過程中,遵循複雜的化學和物理轉換,導致純矽,並推動工程集成電路。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">近年來,儀器儀表配備了便於生產過程的傳感器。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">該數據為生產的高效質量控制提供了機會。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">由於為每一個處理步驟收集的流程數據和復雜的數據分析,</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">該公司可以預測缺陷並識別不合格批次,而不必處理從生產線獲得的昂貴且少量的代表性質量測量。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">所以讓我們來談談</span></span><a href="https://en.wikipedia.org/wiki/Virtual_metrology" style="color: #4d71da; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">虛擬計量學,</span></a><span style="margin: 0px; padding: 0px; vertical-align: inherit;">即從過程數據和</span><span style="font-weight: 700; margin: 0px; padding: 0px;"><em style="margin: 0px; padding: 0px;">異常檢測</em></span><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">(即預測不合格產品</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">的能力)預測產品質量及其端到端特徵</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">的能力。</span></span></div>
<h2 style="clear: both; color: #333333; font-size: 36px; font-stretch: normal; font-weight: normal; line-height: 42px; margin: 0px 0px 10px; padding: 0px; text-align: center;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">預測分析</span></h2>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">即使在倍耐力,任務是開發</span><a href="http://business.pirelli.com/global/en-ww/how-pirelli-is-becoming-data-driven" style="color: #4d71da; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">“智能”生產</span></a><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">系統</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">在整個生產週期收集的數據用於創建預測模型</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">預測分析</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">和</span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">預測分析,</span></span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">並且</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">可以使用</span></span><em style="margin: 0px; padding: 0px;"><span style="font-weight: 700; margin: 0px; padding: 0px;">規範模型</span></em><span style="margin: 0px; padding: 0px; vertical-align: inherit;">來查找設置,以便在的生產週期。</span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">真正的革命是,這些技術至少有一部分現在已經可供中小型企業使用。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">生產數據,原始輸入信息,過程設置和數據可用於實現集成在生產系統中的高級過程控制,以識別異常,包括材料是否適合生產,原因並找到更有效地工作的方法。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;"><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這種方法的一個焦點是數據分析方法提供的工具不需要確定問題。</span><span style="margin: 0px; padding: 0px; vertical-align: inherit;">這些是需要確定用於解決問題的工具的問題。</span></span></div>
<div style="margin-bottom: 10px; padding: 0px;">
<span style="margin: 0px; padding: 0px; vertical-align: inherit;">然而,數據分析過程充滿了陷阱,將數據轉化為解決方案需要時間和創造力。</span></div>
</article></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-82745493086903056622016-11-29T16:07:00.000-08:002016-11-29T16:07:23.583-08:00區塊鏈如何運作?source: https://www.inside.com.tw/2016/09/01/how-does-the-blockchain-work<br />
<div>
<br /></div>
<div>
<br />
<section class=" section--body section--first" style="background-color: white; color: #555555; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px;"><div class="section-content">
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf-after--mixtapeEmbed" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto; padding-left: 30px;">
<strong>原文為《<a class="markup--anchor markup--p-anchor" data-href="https://medium.com/@micheledaliessi/how-does-the-blockchain-work-98c8cd01d2ae#.cjtboz6ge" href="https://medium.com/@micheledaliessi/how-does-the-blockchain-work-98c8cd01d2ae#.cjtboz6ge" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">How Does the Blockchain Work?</a>》。譯者簡志偉,為軟體開發者,譯文刊載於 <a href="https://medium.com/@benzwjian/%E5%8D%80%E5%A1%8A%E9%8F%88%E5%A6%82%E4%BD%95%E9%81%8B%E4%BD%9C-b7c8d4131a0e#.19ln7ais2" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">Medium</a> 。INSIDE 獲授權部分轉載,完整文章請至譯者 <a href="https://medium.com/@benzwjian/%E5%8D%80%E5%A1%8A%E9%8F%88%E5%A6%82%E4%BD%95%E9%81%8B%E4%BD%9C-b7c8d4131a0e#.19ln7ais2" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">部落格</a> 觀看。</strong></div>
<div class="graf--p graf-after--mixtapeEmbed" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
這篇文翻譯自」<a class="markup--anchor markup--p-anchor" data-href="https://medium.com/@micheledaliessi/how-does-the-blockchain-work-98c8cd01d2ae#.cjtboz6ge" href="https://medium.com/@micheledaliessi/how-does-the-blockchain-work-98c8cd01d2ae#.cjtboz6ge" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">How Does the Blockchain Work?</a>」全文。作者 <a class="markup--user markup--p-user" data-action-type="hover" data-action="show-user-card" data-anchor-type="2" data-href="https://medium.com/u/55ec9345778a" data-user-id="55ec9345778a" href="https://medium.com/u/55ec9345778a" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">Michele D'Aliessi</a> 用淺白易懂的文字闡述比特幣 (Bitcoin) 和區塊鏈 (Blockchain) 的運作原理,是一篇很棒的入門文章,因此我決定挑戰翻譯看看,讓更多人了解這個技術。</div>
<div class="graf--p graf-after--mixtapeEmbed" id="b5c7" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
本翻譯文已取得 <a class="markup--user markup--p-user" data-action-type="hover" data-action="show-user-card" data-anchor-type="2" data-href="https://medium.com/u/55ec9345778a" data-user-id="55ec9345778a" href="https://medium.com/u/55ec9345778a" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">Michele D'Aliessi</a> 的同意,全文如下:</div>
<div class="graf--p graf-after--p" id="8ba3" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
自網路問世以來,區塊鏈技術可能是目前為止最棒的發明。它讓我們不用倚靠在無形的信賴或權威機構來做利益交易。舉例來說,我和你打賭 50 元明天舊金山的天氣。我賭它會是晴天,你賭它會是雨天。我們會有三種方式來完成交易:</div>
<ol class="postList" style="font-size: 1em; line-height: 1.75; margin: 1.5em auto; padding-left: 24px;">
<li class="graf--li graf-after--p" id="5de6">我們信賴彼此。不論結果是晴天或雨天,輸家要給贏家 50 元。如果我們是朋友,這會是一個好的交易方式。然而,即便是朋友,也有可能會賴皮不認輸而不願付錢,更何況是陌生人。</li>
<li class="graf--li graf-after--li" id="cb28">我們可以訂定合約,如果有任何一方不願付錢,贏家可以告輸家。但要花錢花時間打官司,只為了討回 50 元,實在是得不償失。</li>
<li class="graf--li graf-after--li" id="6ad9">我們找一個中立的第三者,每人分別先給她 50 元,結果揭曉後,她再把所有的錢 100 元給贏家。無奈的是,這個第三者有可能捲款潛逃。</li>
</ol>
<div class="graf--p graf-after--li" id="fbfc" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
我們無法信任陌生人,也覺得打官司勞神傷財。區塊鏈技術很有趣,因為它幫我們實現第三個交易方式,而且安全、快速和便宜。</div>
<div class="graf--p graf-after--p" id="0c9d" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
我們可以只寫幾行程式碼,讓它執行在區塊鏈網絡 (Blockchain Network) 上,進行交易。以打賭天氣的例子,這支程式會確保 100 元的安全,並且一到明天會自動確認天氣狀況,結果揭曉後,也會自動將 100 元匯到贏家的帳戶裡。在區塊鏈網絡上的交易,是無法被竄改或停止,而且益於大型交易,如賣一間房子或一家公司。</div>
<div class="graf--p graf-after--p" id="de9f" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
<strong class="markup--strong markup--p-strong">本文的目的是不用艱澀的技術用詞來解釋區塊鍊是如何運作,給讀者技術背後擁有的邏輯和機制的基本概念。</strong></div>
<div class="graf--p graf-after--p" id="99d2" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
<a class="markup--anchor markup--p-anchor" data-href="https://en.wikipedia.org/wiki/Bitcoin" href="https://en.wikipedia.org/wiki/Bitcoin" rel="nofollow" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">比特幣</a> 是最為人所知的一項使用區塊鍊技術的應用。電子貨幣可被用來做物品交換,就像美元、歐元、人民幣和其他國家的貨幣。我們先來說明比特幣是如何運作,說明過程中會一點一點帶入區塊鍊的概念。</div>
<blockquote class="graf--blockquote graf--startsWithDoubleQuote graf-after--p graf--last" id="cc7c" style="background-color: transparent; border: 0px; color: #999999; margin: 2.5em auto; padding: 0.65em 0px 0px 3em; position: relative;">
<div style="font-size: 1.728em; line-height: 1.5; margin-bottom: 0.25em; margin-left: auto; margin-right: auto;">
比特幣讓人們第一次可以在網路上交易身家財產,而且是安全的,沒有人可以挑戰其合法性。</div>
</blockquote>
<div style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
<cite>-Marc Andreessen</cite></div>
</div>
</div>
</section><section class=" section--body" style="background-color: white; color: #555555; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 18px;"><div class="section-content">
<div class="section-inner layoutSingleColumn">
<h2 class="graf--h4 graf--leading" id="f090" style="border: 0px; color: #222222; font-size: 1.728em; font-weight: normal; line-height: 1.25; margin: 2.25em 0px 1em; padding: 0px;">
所以,什麼是比特幣?</h2>
<div class="graf--p graf-after--h4" id="983f" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
一塊比特幣就是一個單位的電子比特貨幣 (BTC),並且就像真實的一塊錢貨幣一樣,本身是沒有價值的,只有在進行物品交易時才會產生價值。</div>
<div class="graf--p graf-after--p" id="4ad3" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
在比特幣系統裡,有一本 <a class="markup--anchor markup--p-anchor" data-href="https://en.wikipedia.org/wiki/Ledger" href="https://en.wikipedia.org/wiki/Ledger" rel="nofollow" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">帳本 (ledger)</a>,它是一個電子檔案記錄著所有的交易紀錄。</div>
</div>
<div class="section-inner sectionLayout--outsetColumn">
<br />
<figure class="graf--figure graf--layoutOutsetCenter graf-after--p" id="b5d4" style="margin: 2em auto;"><div class="aspectRatioPlaceholder is-locked">
<div class="aspectRatioPlaceholder-fill">
</div>
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*Serfj4A5sFZzH_gpQk9qJQ.png" data-action="zoom" data-height="514" data-image-id="1*Serfj4A5sFZzH_gpQk9qJQ.png" data-scroll="native" data-width="1400">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="27" style="display: inline-block; vertical-align: baseline;" width="75"></canvas><br />
<figure class="wp-caption aligncenter" style="margin: 2em auto;"><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://inside.com.tw/wp-content/uploads/-000//1/1*Serfj4A5sFZzH_gpQk9qJQ.png" scale="0" src="https://inside.com.tw/wp-content/uploads/-000//1/1*Serfj4A5sFZzH_gpQk9qJQ.png" style="border: 0px; display: block; margin: 2em auto 0.5em; max-width: 100%; padding: 0px;" /><figcaption class="wp-caption-text" style="color: #999999; font-size: 0.69444em; text-align: center;">圖 1. 比特幣電子帳本</figcaption></figure></div>
</div>
</figure></div>
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf-after--figure" id="9943" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
這帳本不是存放在一個中央機構,像是銀行,或是一個資料庫。它擁有無數份副本,散佈存放在區塊鍊網絡上的每一台電腦裡,而每台電腦我們稱為「<a class="markup--anchor markup--p-anchor" data-href="https://en.wikipedia.org/wiki/Node_%28networking%29" href="https://en.wikipedia.org/wiki/Node_%28networking%29" rel="nofollow" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">節點 (node)</a>」。</div>
<div class="graf--p graf-after--p" id="91e7" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
如果 David 想用比特幣轉帳給 Sandra,他就送一個訊息告訴網絡說:他的帳戶減 5 BTC,然後 Sandra 的帳戶加 5 BTC。在網絡中的每個節點都會收到訊息,並且將這筆交易記錄到自己的帳本裡,然後更新帳戶的餘額。</div>
</div>
<div class="section-inner sectionLayout--outsetColumn">
<br />
<figure class="graf--figure graf--layoutOutsetCenter graf-after--p" id="ae97" style="margin: 2em auto;"><div class="aspectRatioPlaceholder is-locked">
<div class="aspectRatioPlaceholder-fill">
</div>
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*O9M0YEX1-_XcF8W0pxSJxg.png" data-action="zoom" data-height="936" data-image-id="1*O9M0YEX1-_XcF8W0pxSJxg.png" data-scroll="native" data-width="1598">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="42" style="display: inline-block; vertical-align: baseline;" width="75"></canvas><br />
<figure class="wp-caption aligncenter" style="margin: 2em auto;"><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://inside.com.tw/wp-content/uploads/-000//1/1*O9M0YEX1-_XcF8W0pxSJxg.png" scale="0" src="https://inside.com.tw/wp-content/uploads/-000//1/1*O9M0YEX1-_XcF8W0pxSJxg.png" style="border: 0px; display: block; margin: 2em auto 0.5em; max-width: 100%; padding: 0px;" /><figcaption class="wp-caption-text" style="color: #999999; font-size: 0.69444em; text-align: center;">圖 2. 請求交易訊息</figcaption></figure></div>
</div>
</figure></div>
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf-after--figure" id="509e" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
說到這裡,關於帳本是由一群電腦共同維護,而不是由一個類似銀行的中心機構來掌管,有三個啟發:</div>
<ul class="postList" style="font-size: 1em; line-height: 1.75; margin: 1.5em auto; padding-left: 24px;">
<li class="graf--li graf-after--p" id="29d2">在銀行系統中,我們只知道自己的交易紀錄和帳戶餘額,而在區塊鍊網絡裡,每個人可以知道任何人的交易紀錄。</li>
<li class="graf--li graf-after--li" id="4d8b">一般來說你信任你的銀行,而比特幣是分布式系統,運行在網路上,任何事情發生錯誤,是沒有客服人員可以幫你的。</li>
<li class="graf--li graf-after--li" id="89c5">區塊鍊不是建構在信賴情感上,其安全性和可靠性是透過特殊的數學函數和程式碼達到的。</li>
</ul>
<blockquote style="background-color: transparent; border: 0px; color: #999999; margin: 2.5em auto; padding: 0.65em 0px 0px 3em; position: relative;">
<div style="font-size: 1.728em; line-height: 1.5; margin-bottom: 0.25em; margin-left: auto; margin-right: auto;">
我們可以定義區塊鍊是一個系統,它讓一群互聯的電腦安全地共同維護一份帳本。</div>
</blockquote>
<div class="graf--p graf-after--pullquote" id="1ad4" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
為了能在區塊鍊網絡裡進行交易,你需要一個 <a class="markup--anchor markup--p-anchor" data-href="https://en.wikipedia.org/wiki/Bitcoin#Wallets" href="https://en.wikipedia.org/wiki/Bitcoin#Wallets" rel="nofollow" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">錢包 (wallet)</a>,它讓你可以存放和交易你的比特幣。只有你可以花費你的比特幣,所以每個錢包被特殊的加密法所保護著,使用一對獨特且配對的鑰匙:公鑰和私鑰,才能解鎖。</div>
<div class="graf--p graf-after--p" id="b795" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
如果一個訊息被公鑰加密,只有配對的私鑰才能解密讀到訊息。反之,如果你用你的私鑰加密訊息,只有配對的公鑰可以解密。所以當 David 想要轉帳,他需要用他的私鑰將轉帳訊息加密後,送到網絡裡,然後每個節點使用 David 的公鑰將訊息解開,以確認是由 David 發送的。</div>
<div class="graf--p graf-after--p" id="e995" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
在加密完成時會產生一個電子簽名,它會被節點們用來確認交易訊息的發送來源和真偽。電子簽名內容是一串文字,它是由交易訊息和私鑰所組成的,所以不能用在其他的交易訊息上。如果你更改交易訊息中任何一個字元,電子簽名也會跟著改變,所以駭客很難更改你的交易訊息或是得知交易金額。</div>
</div>
<div class="section-inner sectionLayout--outsetColumn">
<br />
<figure class="graf--figure graf--layoutOutsetCenter graf-after--p" id="a741" style="margin: 2em auto;"><div class="aspectRatioPlaceholder is-locked">
<div class="aspectRatioPlaceholder-fill">
</div>
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*pRMj7C7wsAWinpE3Yf9LDQ.png" data-action="zoom" data-height="862" data-image-id="1*pRMj7C7wsAWinpE3Yf9LDQ.png" data-scroll="native" data-width="1604">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="40" style="display: inline-block; vertical-align: baseline;" width="75"></canvas><br />
<figure class="wp-caption aligncenter" style="margin: 2em auto;"><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://inside.com.tw/wp-content/uploads/-000//1/1*pRMj7C7wsAWinpE3Yf9LDQ.png" scale="0" src="https://inside.com.tw/wp-content/uploads/-000//1/1*pRMj7C7wsAWinpE3Yf9LDQ.png" style="border: 0px; display: block; margin: 2em auto 0.5em; max-width: 100%; padding: 0px;" /><figcaption class="wp-caption-text" style="color: #999999; font-size: 0.69444em; text-align: center;">圖 3. 電子簽名與加密交易</figcaption></figure></div>
</div>
</figure></div>
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf-after--figure" id="745d" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
錢包的公鑰其實是網絡裡的一個位址 (send to address),所以當你轉比特幣給某人時,你其實是將比特幣轉公鑰的位址。而且你必須證明你是私鑰的所有人,才能進行轉帳。請注意,在網絡裡的交易訊息已經是被加密過的,你不用揭示你的私鑰。</div>
<div class="graf--p graf-after--p" id="2385" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
每個節點都保有一份帳本,但節點是如何知道你的帳戶餘額?區塊鍊系統並沒有記錄每個人的帳戶餘額 (譯注:所以帳本實際上不是像圖 1 一樣),事實上,它只有紀錄網絡上每筆交易紀錄 (如圖 4)。為了得知你的帳戶餘額,你必須分析和驗證所有曾經跟你錢包產生交易的紀錄。</div>
</div>
<div class="section-inner sectionLayout--outsetColumn">
<br />
<figure class="graf--figure graf--layoutOutsetCenter graf-after--p" id="856b" style="margin: 2em auto;"><div class="aspectRatioPlaceholder is-locked">
<div class="aspectRatioPlaceholder-fill">
</div>
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*VlFdSYyVBe46fTcUFLzpHw.png" data-action="zoom" data-height="536" data-image-id="1*VlFdSYyVBe46fTcUFLzpHw.png" data-scroll="native" data-width="1592">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="25" style="display: inline-block; vertical-align: baseline;" width="75"></canvas><br />
<figure class="wp-caption aligncenter" style="margin: 2em auto;"><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://inside.com.tw/wp-content/uploads/-000//1/1*VlFdSYyVBe46fTcUFLzpHw.png" scale="0" src="https://inside.com.tw/wp-content/uploads/-000//1/1*VlFdSYyVBe46fTcUFLzpHw.png" style="border: 0px; display: block; margin: 2em auto 0.5em; max-width: 100%; padding: 0px;" /><figcaption class="wp-caption-text" style="color: #999999; font-size: 0.69444em; text-align: center;">圖 4. 區塊鍊網絡的帳本</figcaption></figure></div>
</div>
</figure></div>
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf--startsWithDoubleQuote graf-after--figure" id="8c98" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
「帳戶餘額」的計算和驗證需要靠之前的交易紀錄。舉個例子,為了轉出 10 BTC 給 John,Mary 先發起一個交易訊息,它包入了之前部分的轉入交易紀錄,只要這些紀錄的轉入金額加總起來剛好或大於 10 BTC 即可發送訊息。這些包入的交易紀錄稱作輸入 (inputs),每個節點會驗證這些輸入的金額加總是等於或大於 10 BTC。這些計算和驗證會由錢包和節點自動完成,使用者不需要煩惱。</div>
</div>
<div class="section-inner sectionLayout--outsetColumn">
<br />
<figure class="graf--figure graf--layoutOutsetCenter graf-after--p" id="bbbf" style="margin: 2em auto;"><div class="aspectRatioPlaceholder is-locked">
<div class="aspectRatioPlaceholder-fill">
</div>
<div class="progressiveMedia js-progressiveMedia graf-image is-canvasLoaded is-imageLoaded" data-action-value="1*pdbhdHDpX2RTF1CG2fbSkg.png" data-action="zoom" data-height="702" data-image-id="1*pdbhdHDpX2RTF1CG2fbSkg.png" data-scroll="native" data-width="1510">
<canvas class="progressiveMedia-canvas js-progressiveMedia-canvas" height="32" style="display: inline-block; vertical-align: baseline;" width="75"></canvas><img class="progressiveMedia-image js-progressiveMedia-image" data-src="https://inside.com.tw/wp-content/uploads/-000//1/1*pdbhdHDpX2RTF1CG2fbSkg.png" scale="0" src="https://inside.com.tw/wp-content/uploads/-000//1/1*pdbhdHDpX2RTF1CG2fbSkg.png" style="border: 0px; display: block; margin: 2em auto 0.5em; max-width: 100%; padding: 0px;" /></div>
</div>
<figcaption class="imageCaption" style="color: #999999; font-size: 0.69444em; text-align: center;">圖 5. 區塊鍊的交易訊息結構</figcaption></figure></div>
<div class="section-inner layoutSingleColumn">
<div class="graf--p graf-after--figure" id="6036" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
至於,系統如何信任這些輸入?它去確認你的錢包在之前所有的轉入交易紀錄中是否真的有這些輸入。為了簡化和加速驗證的過程,每個節點會保留一份特殊的資料來達到目的,也因為這個驗證過程,錢不可能會無緣無故多出來。</div>
<blockquote class="graf--pullquote graf--startsWithDoubleQuote graf-after--p" id="c4d8" style="background-color: transparent; border: 0px; color: #999999; margin: 2.5em auto; padding: 0.65em 0px 0px 3em; position: relative;">
<div style="font-size: 1.728em; line-height: 1.5; margin-bottom: 0.25em; margin-left: auto; margin-right: auto;">
持有比特幣代表的是,帳本上你還未變成輸入的交易紀錄。</div>
</blockquote>
<div class="graf--p graf-after--pullquote graf--last" id="becc" style="font-size: 1em; line-height: 1.85; margin: 1.5em auto;">
在比特幣網絡上執行交易的程式碼都是開源的,這表示任何人只要有電腦和網路就可以進行交易。然而,程式的錯誤有可能導致你的比特幣會不見。還記得嗎?比特幣是分散式網絡,並沒有專屬的客服人員替你找回遺失的錢或錢包密碼。所以你想要用比特幣進行交易,建議使用正式的比特幣錢包軟體 (例如 <a class="markup--anchor markup--p-anchor" data-href="https://bitcoin.org/en/download" href="https://bitcoin.org/en/download" rel="nofollow" style="background-color: transparent; color: #c13b3b; text-decoration: none;" target="_blank">Bitcoin Core</a>),並且妥善保存你的錢包密碼或私鑰。</div>
</div>
</div>
</section></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-40091009091033511592016-01-10T22:09:00.002-08:002016-01-10T22:09:37.779-08:00你不得不知的Cortex-M3和M4微控製器使用秘訣Source-- http://3g.autooo.net/utf8-classid88-id131465.html<br />
http://www.2cm.com.tw/coverstory_content.asp?sn=1311050001<br />
<br />
<h3 style="background-color: white; float: left; font-family: Verdana, Arial, Helvetica, sans-serif; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
你不得不知的Cortex-M3和M4微控製器使用秘訣</h3>
<h6 style="background-color: white; color: #999999; font-family: Verdana, Arial, Helvetica, sans-serif; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
2014-08-08</h6>
<h6 style="background-color: white; color: #999999; font-family: Verdana, Arial, Helvetica, sans-serif; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<br /></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
許多嵌入式開發人員對ARM Cortex處理器架構頗為熟悉,但很少有人能夠對這種流行架構了如指掌,從而可以充分發揮它獨特的特性和性能。ARM Cortex-M4處理器尤為如此,它擁有引以為豪的增強架構、天生的數字信號處理(DSP)能力和可選的浮點加速器,使精於此道的程序設計人員或硬件工程師可以充分發揮它的優勢。本文接下來將就Cortex-M3/M4微控製器(MCU)的一些更有趣的(但經常遭到忽視的)特性展開詳細的論述。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
大部分采用Cortex-M3/M4 MCU的目標應用是便攜式的,並且供電電源來自電池或能源收集係統,因此我們所探討的大部分概念涉及如何減少係統整體能耗的技術。然而,在許多情況下,這些節能技術也是處理器應用設計的有力工具,可提供:</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 更符合成本效益的解決方案</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 更大的升級和采用新特性的設計冗餘</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 有助於產品在激烈競爭市場上脫穎而出的性能和特性</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>ARM Cortex基本介紹</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
就像Advanced RISC Machines(ARM)公司在20世紀80年代所推出的第一代16位處理器內核一樣,ARM Cortex係列以哈佛式RISC架構為基礎,采用適度的矽封裝工藝獲得更高性能,以及代碼和內存效率。該架構在過去十年間大有進展,擴展出了三種不同的子係列,以滿足特定應用的需求:</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● A型係列處理器針對高效能開放應用平台而優化設計。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● R型係列處理器注重提升實時應用的性能和可靠度。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● M型係列處理器特別為采用嵌入式MCU的應用而設計,其性能必須在能源效率和降低解決方案成本之間加以平衡。適用於Cortex M係列的常見應用包括智能電表、人機接口設備、汽車與工業控製係統、白色家電、消費電子產品和醫療器材等。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>Cortex-M3對比Cortex-M4</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
Cortex-M3架構背後的指導思路是設計一種既要滿足應用的成本效益又要提供高性能計算和控製1的處理器。類似的應用包括汽車車身係統、工業控製係統和無線網絡/傳感器產品等。M3係列為32位的ARM處理器架構引進了多項重要特性,包括:</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 不可屏蔽式中斷</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 高度確定性、嵌套、向量式中斷</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 原子位操作</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
● 可選的存儲保護(MPU)</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
除了絕佳的計算性能,Cortex-M3處理器先進的中斷結構還能確保係統迅速響應真實世界的事件,同時仍然提供極低的動態與靜態功耗2。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px; text-align: center;">
<img alt="圖1:Cortex-M3與M4處理器內核的比較。" src="http://www.autooo.net/d/file/autooo/mu/PLD/2014-08-08/64a589a012332d0ee97cbf217999ae76.jpg" style="border-style: solid; border-width: 0px; height: 217px; width: 550px;" /></div>
<div align="center" style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
圖1:Cortex-M3與M4處理器內核的比較。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
Cortex-M3和M4處理器共享許多相同的設計要素,包括先進的片內調試特性,以及執行完整ARM指令集或ARM指令子集(用於THUMB2處理器)的能力。Cortex-M4處理器的指令集具有增強的高效DSP特性庫,包括擴展的單周期16/32位乘法累加器(MAC)、雙16位MAC指令、優化的8/16位SIMD運算及飽和運算指令。總體來說,M3與M4最顯著的差別在於,M4具有可選的單精度(IEEE-754)浮點單元(FPU)。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>多項秘訣造就巧妙解決方案</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
嵌入式設計的成敗經常取決於如何在係統性能、能耗和解決方案成本之間找到適當的平衡。許多情況下,開發人員可以采用Cortex-M處理器上的獨特特性來優化產品成本或能源需求,同時維持、甚至提升它的性能。例如,Cortex-M內核天生的串行I/O能力能夠用於節省能源、簡化開發、釋放外設以用於其它應用任務。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
除了傳統的串行調試(Serial Wire Debug)功能之外,基於ARM Cortex-M的MCU還可以通過它的單引腳串行監視器輸出(Serial Wire Viewer Output,SWO)3提供指令跟蹤接口,如圖2所示。這個接口可以直接把“printf格式的”調試信息傳遞給應用代碼。SWO允許調試信息直接在任何標準的IDE中瀏覽。此外,這些信息也可以用獨立的SWO監視器(例如,Segger的J-Link SWO Viewer軟件4,或是Silicon Labs的energyAware Commander 4)進行瀏覽。由於SWO輸出內建於內核硬件本身,因此它是Cortex-M內核與生俱來的優點。SWO不占用MCU的任何UART接口,這些接口它們可能早已被分配給了應用。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px; text-align: center;">
<img alt="圖2:專用ARM Cortex SWO接口節省I/O引腳並加速調試。" src="http://www.autooo.net/d/file/autooo/mu/PLD/2014-08-08/595598e88d518873c51a026d3e43f870.jpg" style="border-style: solid; border-width: 0px; height: 230px; width: 550px;" /></div>
<div align="center" style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
圖2:專用ARM Cortex SWO接口節省I/O引腳並加速調試。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
基於SWO的調試還有一個重要的優勢在於,它讓微控製器在進入最低的休眠模式時,保持調試連接有效,而在大多數情況下,傳統的調試連接這時是不能正常工作的。SWO的指令追蹤還可以用於跟蹤程序計數器,以幫忙IDE統計出程序各項功能所占用的時間。這些統計數字能夠與電流測量結合起來,幫助開發人員對設計功耗進行微調。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
基於Cortex-M的微控製器供應商正在開始重新認識這項優點,而且有些廠商已經為了這個目的而把功耗模式和電流測量硬件納入到本身的開發平台。例如,Silicon Labs的EFM32 Gecko MCU入門級和開發級工具包都包含功耗測量輸出,並可搭配energyAware Profiler工具6中的程序代碼追蹤功能。圖3顯示了如何讓設計人員精確定位到哪個程序功能塊最耗費能源,並且能夠快速調試其它與能源有關的問題。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px; text-align: center;">
<img alt="圖3:軟硬件工具精確定位耗能最大的功能,無需示波器和萬用表,快速排除問題。" src="http://www.autooo.net/d/file/autooo/mu/PLD/2014-08-08/b962407763143fd830934652d38f8166.jpg" style="border-style: solid; border-width: 0px; height: 457px; width: 550px;" /></div>
<div align="center" style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
圖3:軟硬件工具精確定位耗能最大的功能,無需示波器和萬用表,快速排除問題。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>智能休眠節省每一微瓦</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
ARM Cortex-M處理器的Sleep-on-Exit(中斷完成時直接進入休眠)是另一項“一箭雙雕”的功能,可同時節省CPU周期和能耗。這點在由中斷所驅動的應用中格外有用,因為處理器的大部分時間不是在執行中斷處理,就是在中斷事件之間休眠。在進入中斷服務例程(ISR)時,MCU必須花費好幾個指令周期把當前線程狀態入棧,然後在退出中斷處理返回時恢複原有線程狀態,即“出棧”。當應用需要處理器在退出ISR後直接進入休眠狀態時,傳統MCU仍然必須恢複原先存儲的狀態信息,然後線程代碼才能讓MCU進入休眠狀態。同樣地,當下次的中斷喚醒MCU時,它的狀態必須再次入棧。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
而當使能ARM Cortex-M微控製器上的Sleep-on-Exit功能後,MCU就會在中斷處理完成後直接進入休眠狀態,而不用先返回到原有線程上(見圖4)。這會使處理器仍然保持在中斷狀態,因為消除了喚醒再入棧過程,因而節省下許多寶貴的機器周期。消除入棧出棧過程既節省了時間也節省了能耗,否則電能就會被不必要的指令周期白白消耗,也包括哪些傳統MCU在休眠和喚醒之間管理堆棧的代碼。而且,當處理器被中止調試請求(Halt Debug Request)喚醒時,出棧過程將會自動進行。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px; text-align: center;">
<img alt="ARM Cortex-M處理器" src="http://www.autooo.net/d/file/autooo/mu/PLD/2014-08-08/6565d9f762e03276034338dbb0b305fc.jpg" style="border-style: solid; border-width: 0px; height: 320px; width: 282px;" /></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px; text-align: center;">
<img alt="ARM Cortex-M的Sleep-on-Exit功能通過避免不必要的代碼執行和減少出棧入棧操作降低功耗。" src="http://www.autooo.net/d/file/autooo/mu/PLD/2014-08-08/eb77a89327e379e5d573a86a3cd4e249.jpg" style="border-style: solid; border-width: 0px; height: 171px; width: 550px;" /></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
圖4:ARM Cortex-M的Sleep-on-Exit功能通過避免不必要的代碼執行和減少出棧入棧操作降低功耗。(引自:《The Definitive Guide to the ARM Cortex-M31》)</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>ARM Cortex-M4運行更快、休眠功耗更低</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
像許多MCU一樣,Cortex-M3/4處理器通常能夠采用高時鍾速率的方法在中斷驅動的應用中節省能耗。如果處理器大部分時間處於休眠狀態,這種看似違背直覺但普遍采用的節能策略就會很好,因為運行時間減少所節省的能耗遠遠大於稍高的操作電流。簡單來說,多花10%的電可以省掉20%的時間,總體來說是節能了。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
這種技術可以應用在任何Cortex-M係列的處理器上,而涉及密集運算任務的應用也能從Cortex-M4處理器的額外能力中受益。它的單周期DSP指令和可選的浮點加速器能大大減少諸如數字信號處理、過濾、分析或波形合成等功能所需要的執行周期數。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
一些應用僅僅需要DSP處理能力。例如,有些安全係統采用一種以聲學分析來感測玻璃破損的裝置。玻璃破損時會發出一連串獨特的聲音和振動,並且在玻璃特有的固有頻率時達到最大,在這個例子中是13kHz。大多數采用傳感器接口的係統隻有在所監測的頻率被監測到時,才喚醒處理器。但是當設計中使用帶DSP功能的Cortex-M4時就能額外節能,因為它在執行實際的玻璃破損分析時比軟件解決方案更快。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
甚至,這些使用基於M4微控製器的應用可以更加節能,因為MCU中所包含的高級休眠模式和自治外設可以在CPU休眠時執行許多日程任務。例如,以Cortex-M4為內核的Wonder Gecko MCU7具有五種不同的低功耗模式,包括20nA的關機狀態和950nA的深度休眠模式(實時時鍾有效、RAM和寄存器內容保持、使能掉電檢測)。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
上麵提及的節能特性也能帶來其它優勢。例如,在超音波/聲學水表之類的應用中,它們必須在小電池供電下運行多年,需要MCU盡可能長的保持在休眠狀態。除了有助於減少MCU喚醒時間之外,Cortex-4 DSP和浮點算術指令也能使用成熟的濾波功能從廉價聲學傳感器輸出中獲得所需的信息,從而避免采用昂貴的超聲波流量傳感器。在這個應用實例中,Wonder Gecko MCU的外設還能夠作為模擬狀態機提供額外的能量節省,它僅僅在需要時才喚醒Cortex-M4處理器。</div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong> </strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
<strong>結論</strong></div>
<div style="color: black; font-size: 16px; font-weight: normal; line-height: 22.4px; margin-top: 5px; padding-left: 8px; padding-right: 8px;">
雖然並不完備,但這些林林總總的秘訣與妙方應該能讓各位產生好的思路,可以在下一次設計中充分利用Cortex-M係列中一些較不為人知的特性所帶來的好處</div>
</h6>
<ins class="adsbygoogle" data-ad-client="ca-pub-1158519956220368" data-ad-slot="4053896053" data-adsbygoogle-status="done" style="background-color: white; display: inline-block; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 16px; height: 50px; line-height: 22.4px; width: 320px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 50px; margin: 0px; padding: 0px; position: relative; visibility: visible; width: 320px;"></ins></ins>Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-25739667390780129722016-01-10T22:09:00.001-08:002016-01-10T22:09:28.356-08:00麻雀雖小 五臟俱全:MCU專用RTOS簡述<span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">麻雀雖小 五臟俱全:MCU專用RTOS簡述</span><span style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;"><br /><br />DIGITIMES中文網 原文網址: <a href="http://www.digitimes.com.tw/tw/dt/n/shwnws.asp?CnlID=13&packageid=9376&id=0000424643_EE45QU335SXGR42FQO53O&cat=10&ct=1#ixzz3wuesZBnC" style="color: #003399; text-decoration: none;">麻雀雖小 五臟俱全:MCU專用RTOS簡述</a> <a href="http://www.digitimes.com.tw/tw/dt/n/shwnws.asp?CnlID=13&packageid=9376&id=0000424643_EE45QU335SXGR42FQO53O&cat=10&ct=1#ixzz3wuesZBnC" style="color: #003399; text-decoration: none;">http://www.digitimes.com.tw/tw/dt/n/shwnws.asp?CnlID=13&packageid=9376&id=0000424643_EE45QU335SXGR42FQO53O&cat=10&ct=1#ixzz3wuesZBnC</a></span><br />
<br />
<span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">微控制器(MCU)廣泛應用在各行各業,如各式家電、工業自動化,即時控制、資料採集等領域,為因應工控所需的即時(Realtime)控制、快速回應等需求,因此MCU大多搭載RTOS(即時作業系統)運作。隨著物聯網的興起,軟體業也為RTOS加入物聯網的成分,以提早卡位物聯網的核心軟體市場…</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><img src="http://www.digitimes.com.tw/tw/x/img/x.gif" style="border: none; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><strong style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">各種處理器專用之OS</strong><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">在一般功能(General-purpose)的處理器市場分類中,若以功能與執行速度來說,大致分為CPU > MPU > MCU。CPU的功能最強,主要應用在電腦產品;MPU功能次之,其應用多元,主要應用在嵌入式系統與精簡型電腦等多種;而MCU則是以單一應用為主,應用在各式家電、電子產品、嵌入式產品、穿戴式裝置、物聯網(IoT)應用產品等控制應用。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br />
<div id="newsphote" style="clear: both; float: left; font-family: 微軟正黑體; font-size: 8pt; height: 799.872px; letter-spacing: 1.5pt; line-height: 12px; margin-right: 12px; padding: 3px 0px; width: 250px;">
<div align="right" style="cursor: pointer; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-1-QO53O.jpg?2016111135922" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=download_file('http://mms.digitimes.com/source/2015/424643-1-qo53o.ppt'); style=color:#ffffff;cursor:pointer;>下載原始數據</a> | <a onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-1-QO53O.jpg','IMG',config='height=665,width=430,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>1/3</span><br> "><div style="float: right; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: right; width: 250px;">
<img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" /><span style="cursor: pointer;">放大</span></div>
<img border="0" src="http://mms.digitimes.com/imageCH_4.aspx?img=/NewsImg/2015/0507/424643-1-QO53O.jpg&W=250&2016111135922" style="border: none;" title="放大圖片" /></a></div>
<table border="0" cellpadding="0" cellspacing="0"><tbody>
<tr><td style="font-size: 16px; line-height: 18pt;" width="100%"></td></tr>
</tbody></table>
<div align="right" style="cursor: pointer; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-2-QO53O.jpg?2016111135922" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=download_file('http://mms.digitimes.com/source/2015/424643-2-qo53o.xls'); style=color:#ffffff;cursor:pointer;>下載原始數據</a> | <a onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-2-QO53O.jpg','IMG',config='height=989,width=630,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>2/3</span><br> "><div style="float: right; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: right; width: 250px;">
<img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" /><span style="cursor: pointer;">放大</span></div>
<img border="0" src="http://mms.digitimes.com/imageCH_4.aspx?img=/NewsImg/2015/0507/424643-2-QO53O.jpg&W=250&2016111135922" style="border: none;" title="放大圖片" /></a></div>
<table border="0" cellpadding="0" cellspacing="0"><tbody>
<tr><td style="font-size: 16px; height: 1px; line-height: 18pt; padding: 0px; width: 20px;"> <span style="border-top-color: rgb(249, 170, 21); border-top-style: dotted; border-top-width: 2px; width: auto;"> </span></td></tr>
<tr><td style="font-size: 16px; line-height: 18pt;" width="100%"></td></tr>
</tbody></table>
<div align="right" id="newsphote_more" style="font-size: 10pt; line-height: 14pt; margin-bottom: 4px; margin-top: 4px; overflow: hidden; text-align: justify; width: 250px;">
<b><div style="float: right; font-size: 9pt; line-height: 14pt; margin: 4px 0px; overflow: hidden; text-align: right; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-3-QO53O.jpg" id="pic_alink" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;>Contiki OS適用於IoT的WSN應用,圖為各感測器資訊傳遞路經的模擬測試。<br><br><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-3-QO53O.jpg','IMG',config='height=570,width=630,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>3/3</span><br> "><img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" />更多</a></div>
</b></div>
<div id="pos1" style="font-size: 8pt; height: 1px; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
</div>
</div>
<span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">MCU內部整合了KHz~MHz級的CPU、KB~MB級的記憶體單元(RAM與ROM/EEPROM/Flash)、時脈產生器(Oscillator;Clock Generator)、與I/O擴充單元等,可視為一種速度較慢的系統單晶片(SoC)。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">由於內部記憶體容量小,因此大型作業系統如Windows、Linux等是不可能塞入MCU去執行的,且MCU大多被應用在即時控制的環境,因此許多容量小的RTOS(Real-Time Operating System;即時作業系統),便成為開發MCU軟體的主要平台。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><img src="http://www.digitimes.com.tw/tw/x/img/x.gif" style="border: none; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><strong style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">主打嵌入式應用的中高階RTOS</strong><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">RTOS的種類繁多,主要設計給基於MPU或MCU的嵌入式系統所使用。例如MPU等級專用的有Integrity、QNX、VxWorks等功能強大之RTOS;至於體積較小巧,主要支援MCU等級為主的RTOS,則有Nucleus、ThreadX、Unison OS、ucOS II/III等等。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">以Green Hills Software推出的Integrity OS為例,就是一種支援MPU (甚至CPU等級)為主的RTOS。其強項在於Integrity-178版本已通過EAL 6+?(資訊安全)認證與DO-178B(飛安環境) A級認證,被應用在極度重視安全和可靠性的市場,例如戰鬥機(如B-2、F-16、F-22、F-35)與民航機(如Airbus A380)等領域。該RTOS支援ARM、XScale、Blackfin、Freescale (已併入NXP) ColdFire、MIPS、PowerPC、AMD x86(嵌入式APU)等CPU/MPU平台。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">另一個知名的QNX RTOS,採用微核心架構,是唯一成功打入商用市場的OS,其強項是多媒體的即時處理能力,適用於車(機)上娛樂裝置與手機等嵌入式市場。QNX於2010年被BlackBerry購併,並開發出BB 10作業系統。QNX支援IA32、MIPS、PowerPC、SH-4、ARM、StrongARM、XScale等CPU/MPU平台。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">至於像是IntervalZero的RTX、RTX64,則是設計來與微軟Windows共存共容的RTOS,搭配EtherCAT協定來做為工廠自動化的應用。其中,Windows主要負責GUI、儲存、運算,RTX則負責即時工控與資料採集,讓工控軟體開發更容易。以上的RTOS都是MB至GB等級的MPU等級OS,不適用於MCU的環境。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><img src="http://www.digitimes.com.tw/tw/x/img/x.gif" style="border: none; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><strong style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">主打MCU應用的商用RTOS</strong><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">中低階RTOS部分,主要是把軟體功能極盡精簡到MB甚至KB等級,使整個OS與主要應用程式,均可以塞入MCU裡的ROM/EEPROM/Flash。由於MCU應用的領域更加廣泛,其軟體必須力求更加精簡,因此MCU專用的RTOS大多具備非常高度模組化的架構,從核心、驅動程式、檔案系統、週邊I/O、網路支援等,都可以量身訂作,以利產品快速上市。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">商用的RTOS有些會提供原始碼給授權客戶,而開源的RTOS則更能自由使用,讓開發人員可以編譯出程式碼最小、最佳化的執行環境。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">由於各晶片廠所推出的MCU產品/開發板,都會有其對應的OS與IDE(整合軟體開發環境),但這些OS與軟體開發環境可能只適用於該廠的MCU產品,因此第三方軟體廠商,就開發出跨晶片/跨硬體平台的OS與IDE,讓開發人員不須因為換了硬體平台,軟體就必須全部改寫。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">目前MCU OS/IDE市場佔有率最高的,大多是軟體公司所推出商用RTOS(搭配各廠商的MCU產品),然隨著ARM推出Cortex-M、Cortex-R等指令集架構,進軍穿戴式與物聯網應用市場,使得ARM架構(採開源碼)的RTOS開始有提升的趨勢。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">Mentor Graphics旗下Accelerated Technology公司所推出的Nucleus,採Microkernel設計,號稱有30億個裝置導入,優勢是核心長度可以小至2KB,且開發人員不需要撰寫嵌入式裝置專用BSP(開發板支援套裝軟體),因此被廣泛應用到消費性電子、行動裝置、車用電子、智慧能源、醫療儀器、工業/工控等領域。</span><span style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;"><br /></span><br />
<span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">早期採用聯發科MT6217晶片的大陸山寨、白牌、雙卡2G手機,就是執行Nucleus RTOS。該RTOS支援ARM、MicroBlaze、MIPS、Nios II、Power、SuperH、XScale等嵌入式MCU架構。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">Express Logic推出的ThreadX,則是一套免收權利金的RTOS,其優點是具備超快速的開機時間、反應時間,其Picokernel核心長度低於2KB,並通過安全規範,號稱有21億個裝置導入使用。例如HP的旗下印表機和事務機便採用該RTOS。可廣泛支援各式32位元MCU,包含ARM、Atmel、BlackFin、CoreFire/68K、EFM32、Freescale (NXP)、FM3、H8、XMC、M-Core、MicroBlaze、MIPS、Nios II、Power、STM32、StrongARM、Synopsys ARC、TI、Win32、x86/x386、XScale等等。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br />
<div id="newsphote" style="clear: both; float: left; font-family: 微軟正黑體; font-size: 8pt; height: 799.872px; letter-spacing: 1.5pt; line-height: 12px; margin-right: 12px; padding: 3px 0px; width: 250px;">
<div align="right" style="cursor: pointer; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-1-QO53O.jpg?201611114318" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=download_file('http://mms.digitimes.com/source/2015/424643-1-qo53o.ppt'); style=color:#ffffff;cursor:pointer;>下載原始數據</a> | <a onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-1-QO53O.jpg','IMG',config='height=665,width=430,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>1/3</span><br> "><div style="float: right; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: right; width: 250px;">
<img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" /><span style="cursor: pointer;">放大</span></div>
<img border="0" src="http://mms.digitimes.com/imageCH_4.aspx?img=/NewsImg/2015/0507/424643-1-QO53O.jpg&W=250&201611114318" style="border: none;" title="放大圖片" /></a></div>
<table border="0" cellpadding="0" cellspacing="0"><tbody>
<tr><td style="font-size: 16px; line-height: 18pt;" width="100%"></td></tr>
</tbody></table>
<div align="right" style="cursor: pointer; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-2-QO53O.jpg?201611114319" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=download_file('http://mms.digitimes.com/source/2015/424643-2-qo53o.xls'); style=color:#ffffff;cursor:pointer;>下載原始數據</a> | <a onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-2-QO53O.jpg','IMG',config='height=989,width=630,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>2/3</span><br> "><div style="float: right; font-size: 8pt; line-height: 14pt; overflow: hidden; text-align: right; width: 250px;">
<img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" /><span style="cursor: pointer;">放大</span></div>
<img border="0" src="http://mms.digitimes.com/imageCH_4.aspx?img=/NewsImg/2015/0507/424643-2-QO53O.jpg&W=250&201611114318" style="border: none;" title="放大圖片" /></a></div>
<table border="0" cellpadding="0" cellspacing="0"><tbody>
<tr><td style="font-size: 16px; height: 1px; line-height: 18pt; padding: 0px; width: 20px;"> <span style="border-top-color: rgb(249, 170, 21); border-top-style: dotted; border-top-width: 2px; width: auto;"> </span></td></tr>
<tr><td style="font-size: 16px; line-height: 18pt;" width="100%"></td></tr>
</tbody></table>
<div align="right" id="newsphote_more" style="font-size: 10pt; line-height: 14pt; margin-bottom: 4px; margin-top: 4px; overflow: hidden; text-align: justify; width: 250px;">
<b><div style="float: right; font-size: 9pt; line-height: 14pt; margin: 4px 0px; overflow: hidden; text-align: right; width: 250px;">
<a href="http://mms.digitimes.com/NewsImg/2015/0507/424643-3-QO53O.jpg" id="pic_alink" rel="lightbox[g1]" style="color: #1e4061; text-decoration: none;" title="<span style=float:left;margin-left:5px;width:490px;>Contiki OS適用於IoT的WSN應用,圖為各感測器資訊傳遞路經的模擬測試。<br><br><a style=float:left;padding-bottom:10px;color:#ffffff;cursor:pointer; onclick=window.open('/tw/showimg_fusion2-1.asp?news_key=424643&filename=424643-3-QO53O.jpg','IMG',config='height=570,width=630,scrollbars=no'); style=color:#ffffff;cursor:pointer;>預覽列印</a></span><span style=float:right;>3/3</span><br> "><img align="absmiddle" hspace="3" src="http://mms.digitimes.com/tw/x/img/enlarge_icon.gif" style="border: none;" />更多</a></div>
</b></div>
<div id="pos1" style="font-size: 8pt; height: 1px; line-height: 14pt; overflow: hidden; text-align: justify; width: 250px;">
</div>
</div>
<span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">Wind River公司所推出的VxWorks,主要針對嵌入式系統設計,採Monolithic (單體式)核心,優勢是具備先佔式多工處理核心、循環執行、岔斷快速反應等特性,原生支援64位元處理器架構(x64)、可進行平行(SMP)/非平行(AMP)處理,累積至今有超過15億個裝置導入。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">新版VxWorks 7則瞄準IoT所需要的可擴充性、安全性、連結性、繪圖能力、虛擬化等做強化,而全功能的VxWorks微核心長度只要20KB。VxWorks廣受科技業界的採用,登陸火星的Curiosity(好奇號)便採用VxWorks。該RTOS支援Intel x86(包含Quark SoC與x86-64)、MIPS、PowerPC、SH-4、ARM等CPU/MPU架構。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">RoweBots公司的Unison OS,則是一款完全相容於POSIX(可移植作業系統介面)的RTOS,適用於MCU、DSC、DSP、SoC、FPGA等32位元的硬體開發環境,其好處是特別針對物聯網的應用,提升其系統安全性,且核心程式碼在某些應用架構可以低到僅1KB。支援Microchip PIC32、Renesas R32C/SH2A、ST STM32、TI ARM Cortex-M3等32位元MCU。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">Micrium的μc/OS-II (microcontroller OS version 2),主打可攜、能在ROM執行、彈性、先佔式多工的RTOS核心,可管理高達250個應用任務。μc/OS-III則主打無限應用任務、幾近於零的岔斷,並可提供原始碼給客戶。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">其優勢在於該系統原始碼開放、整潔一致、註釋詳盡,亦通過FAA認證與DO-178B認證,適合各種嵌入式與物聯網的系統開發,核心大小從5或6KB~24KB。至於μc/OS-III HW-RTOS,則是針對ARM Cortex-M為主的MCU做硬體加速。該RTOS可支援超過100種DSP、MPU、MCU。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><img src="http://www.digitimes.com.tw/tw/x/img/x.gif" style="border: none; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><strong style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">ARM MCU促使開源RTOS興起</strong><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">近年來由於ARM架構的處理器橫掃全球智慧行動裝置(手機/平板)市場,除了搭配各MCU/MPU硬體平台所推出的商用RTOS/IDE之外,為進軍物聯網與穿戴式的MCU級應用,ARM推出Cortex-M與Cortex-R的指令集架構,搭配開源的OS/IDE來搶佔MCU的應用市場。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">例如ARM推出的mbed OS與相關開發環境,便著重於嵌入式裝置與IoT的應用,具備連接性、高效率、安全性、生產力的OS,搭配其mbed-rtos函式庫,亦可做為RTOS的應用。該mbed開發環境,可開發出智慧家庭、智慧城市、穿戴式等應用產品。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">此外,坊間針對ARM平台所推出的開源RTOS/IDE很多,例如FreeRTOS、uKOS-II、Atomthreads、BeRTOS社群版、ChibiOS/RT、CoActionOS、eCos、Embox、Erika Enterprise/RT-Druid、Keil (ARM) RTX、Lepton、nOS、Nut/OS、NuttX、RIOT、RT-Thread、TI-RTOS-KERNEL(SYS/BIOS)、TNeo等等,讓開發人員有更多的選擇。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><img src="http://www.digitimes.com.tw/tw/x/img/x.gif" style="border: none; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><strong style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">其他專用MCU的非即時OS概述</strong><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">此外,也有許多針對MCU設計的開源OS (非RTOS),但同樣具有體積小的特性,有些是針對IoT的WSN(無線感測網路)應用,例如Contiki OS、TinyOS。而有些則具備一般桌上型圖形化使用介面(GUI),例如SymbOS、Wheels OS等。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">Contiki OS是一套開源的微型OS,可應用在Atmel ARM/AVR、LPC、PIC32、TI MSP430/CC2430/2538/2630/2650、STM32W等MCU做IoT應用,也可在博物館級的8位元電腦(Apple II、Atari、Commodore等)做上網連線、甚至在骨灰級遊樂器(Atari Jaguar、Game Boy/Advance、GP32、任天堂紅白機、PC Engine等)上執行。</span><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><br style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;" /><span style="background-color: white; font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;">至於SymbOS,則是一套能在8位元Z80 CPU (如MSX、Amstrad)的古董電腦上執行之免費多媒體圖形作業系統,賦予如Windows 95般的操作畫面,讓舊電腦回春。</span><span style="font-family: 微軟正黑體; font-size: 16px; letter-spacing: 1.33333px;"><br /><br /></span>Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-55844089608963068522015-06-24T23:25:00.002-07:002015-06-24T23:25:48.027-07:00Hadoop vs small size files<h3 style="background-color: #fafafa; font-family: 'ms song', 'Microsoft Yahei', SimHei, Verdana, Helvetica, SimSun, Arial, 'Arial Unicode MS', MingLiu, PMingLiu, 'MS Gothic', sans-serief; margin: 0px; padding: 0px; text-align: center;">
HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS)</h3>
<div>
http://www.open-open.com/lib/view/1330605869374</div>
<div>
<br /></div>
<div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
一、概述</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
手机图片或者像淘宝这样的网站中的产品图片特点:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1)、大量手机用户同时在线,执行上传、下载、read等图片操作</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2)、文件数量较大,大小一般为几K到几十K左右</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
HDFS存储特点:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1) 流式读取方式,主要是针对一次写入,多次读出的使用模式。写入的过程使用的是append的方式。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2) 设计目的是为了存储超大文件,主要是针对几百MB,GB,甚至TB的文件</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(3) 该分布式系统构建在普通PC机组成的集群上,大大降低了构建成本,并屏蔽了系统故障,使得用户可以专注于自身的操作运算。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
HDFS与小图片存储的共通点和相悖之处:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1) 都建立在分布式存储的基本理念之上</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2) 均要降低成本,利用普通的PC机构建系统集群</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1) HDFS不适合大量小文件的存储,因namenode将文件系统的元数据存放在内存中,因此存储的文件数目受限于 namenode的内存大小。HDFS中每个文件、目录、数据块占用150Bytes。如果存放1million的文件至少消耗300MB内存,如果要存 放1billion的文件数目的话会超出硬件能力</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2) HDFS适用于高吞吐量,而不适合低时间延迟的访问。如果同时存入1million的files,那么HDFS 将花费几个小时的时间。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(3) 流式读取的方式,不适合多用户写入,以及任意位置写入。如果访问小文件,则必须从一个datanode跳转到另外一个datanode,这样大大降低了读取性能。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
二、HDFS文件操作流程</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " src="http://static.open-open.com/lib/uploadImg/20120301/20120301204259_984.gif" style="border: 0px; height: 458px; margin: 0px; padding: 0px; width: 688px;" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
reading:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="384" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204300_514.gif" style="border: 0px; margin: 0px; padding: 0px;" width="729" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
writing:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="391" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204300_61.gif" style="border: 0px; margin: 0px; padding: 0px;" width="751" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
三、HDFS自带的小文件存储解决方案</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
对于小文件问题,hadoop自身提供了三种解决方案:Hadoop Archive、 Sequence File 和 <a href="https://www.blogger.com/null" name="OLE_LINK2" style="color: #22749b; margin: 0px; padding: 0px;"></a><a href="https://www.blogger.com/null" name="OLE_LINK1" style="color: #22749b; margin: 0px; padding: 0px;">CombineFileInputFormat</a></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1) Hadoop Archive</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="318" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204301_516.gif" style="border: 0px; margin: 0px; padding: 0px;" width="334" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
归档为bar.har文件,该文件的内部结构为:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="384" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204301_557.gif" style="border: 0px; margin: 0px; padding: 0px;" width="678" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
创建存档文件的问题:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
1、存档文件的源文件目录以及源文件都不会自动删除需要手动删除</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
2、存档的过程实际是一个mapreduce过程,所以需要需要hadoop的mapreduce的支持</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
3、存档文件本身不支持压缩</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
4、存档文件一旦创建便不可修改,要想从中删除或者增加文件,必须重新建立存档文件</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
5、创建存档文件会创建原始文件的副本,所以至少需要有与存档文件容量相同的磁盘空间</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2) Sequence File</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
sequence file由一系列的二进制的<key style="margin: 0px; padding: 0px;" value="">对组成,其中key为小文件的名字,value的file content。</key></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="398" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204301_189.gif" style="border: 0px; margin: 0px; padding: 0px;" width="693" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
创建sequence file的过程可以使用mapreduce工作方式完成</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
对于index,需要改进查找算法</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
对小文件的存取都比较自由,也不限制用户和文件的多少,但是该方法不能使用append方法,所以适合一次性写入大量小文件的场景</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(3) CombineFileInputFormat</div>
<div align="left" style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
CombineFileInputFormat是一种新的inputformat,用于将多个文件合并成一个单独的split,另外,它会考虑数据的存储位置。</div>
<div align="left" style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
该方案版本比较老,网上资料甚少,从资料来看应该没有第二种方案好。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
四、WebGIS解决方案概述</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
在地理信息系统中,为了方便传输通常将数据切分为KB大小的文件存储在分布式文件系统中,论文结合WebGIS数据的相关特征,将相邻地理位置的小 文件合并成一个大的文件,并为这些文件构建索引。论文中将小于16MB的文件当做小文件进行合并处理,将其合并成64MB的block并构建索引。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " src="http://static.open-open.com/lib/uploadImg/20120301/20120301204302_917.gif" style="border: 0px; height: 362px; margin: 0px; padding: 0px; width: 699px;" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
从以上索引结构和文件存储方式可以看出,index是一般的定长hash索引,并且采用的是存储全局index文件的方式</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
read的过程是将小文件append到下文件后边,然后更新索引的过程</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
delete文件的过程采用lazy模式,更改的是FVFlag,在空间重新分配的过程中,才会根据该flag删除文件。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
五、BlueSky解决方案概述</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
BlueSky是中国电子教学共享系统,主要存放的教学所用的ppt文件和视频文件,存放的载体为HDFS分布式存储系统。在用户上传PPT文件的 同时,系统还会存储一些文件的快照,作为用户请求ppt时可以先看到这些快照,以决定是否继续浏览,用户对文件的请求具有很强的关联性,当用户浏览ppt 时,其他相关的ppt和文件也会在短时间内被访问,因而文件的访问具有相关性和本地性。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
paper主要提出了两个基本观点:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(1) 将属于同一课件的小文件合并成一个大文件,从而减轻namenode的压力,提高小文件的存储效率</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(2) 提出了一种两级预取机制以提高小文件的读取效率,(索引文件预取和数据文件预取)索引文件预取是指当用户访问某个文件时,该文件 所在的block对应的索引文件被加载到内存中,这样,用户访问这些文件时不必再与namenode交互了。数据文件预取是指用户访问某个文件时,将该文 件所在课件中的所有文件加载到内存中,这样,如果用户继续访问其他文件,速度会明显提高。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
BlueSky上传文件的过程:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="384" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204302_859.gif" style="border: 0px; margin: 0px; padding: 0px;" width="720" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
BlueSky阅览文件的过程:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="401" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204302_486.gif" style="border: 0px; margin: 0px; padding: 0px;" width="727" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<strong style="margin: 0px; padding: 0px;">文件合并:</strong></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
文件合并过程如果合并之后文件的大小小于block64MB的大小则直接存放到一个block中。(合并之后的文件包括local index文件)</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
如果合并之后的文件大小大于64MB有两种方式split这个大文件:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
1、 local index文件、ppt文件、standresolution picture series存放在一个block中,剩下的picture series存在在其他的block中。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
2、 在相邻block的连接处填充空白文件,具体过程:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="309" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204303_211.gif" style="border: 0px; margin: 0px; padding: 0px;" width="393" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
文件映射:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
文件的命名方式,分离的预取图片有其自身的命名方式,具体见paper。文件映射过程中,除了block中的局部索引文件之外,还有一个全局映像文 件。该文件存放的内容为<original block="" file="" merged="" name="" number="" style="margin: 0px; padding: 0px;"></original></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
根据全局mapping table 就可以根据merged file name 和 block Id到namenode上得到datanode的信息,然后到根据<datanodeinfo block="" style="margin: 0px; padding: 0px;">到具体的机器上找到相应的block获取到localindex file,根据original file name从local index file中查到<file length="" offset="" style="margin: 0px; padding: 0px;">从而定位到data。根据预取策略,在此过程中也会预取到local index file 和相关的file</file></datanodeinfo></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
六、facebookHayStack解决方案概述</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="456" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204303_118.gif" style="border: 0px; margin: 0px; padding: 0px;" width="447" /><img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="467" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204303_277.gif" style="border: 0px; margin: 0px; padding: 0px;" width="458" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
haystack是一个不同于HDFS的分布式系统,如果想在HDFS的基础上构建小文件存储系统,个人认为可以参考借鉴其索引结构的设计。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
1、 directory 中有logical volume id<->physicalvolume id。根据<key alternative="" key="" style="margin: 0px; padding: 0px;">可以通过directory拼出来<a href="http://%3Ccdn%3E/%3cchache%3e/%3cmachine" style="color: #005fa9; margin: 0px; padding: 0px; text-decoration: none;">http://<cdn style="margin: 0px; padding: 0px;">/<chache style="margin: 0px; padding: 0px;">/<machine a="" style="margin: 0px; padding: 0px;">id>/ <logical id="" photo="" style="margin: 0px; padding: 0px;" volume="">。 因此在directory端存在着<logicalvolume id="" machine="" style="margin: 0px; padding: 0px;">映射以及<key alternative="" key="" logical="" style="margin: 0px; padding: 0px;" volume="">映射</key></logicalvolume></logical></machine></chache></cdn></a></key></div>
<a href="http://%3Ccdn%3E/%3cchache%3e/%3cmachine" style="background-color: #fafafa; color: #005fa9; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin: 0px; padding: 0px; text-decoration: none;"><div style="color: #404040; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
2、 根据url到store端之后,可以根据logicalvolume id获得相应的physical volume的位置,然后physical中存在super block,根据映射<key alternative="" flag="" key="" offset="" size="" style="margin: 0px; padding: 0px;">可以得到photo数据</key></div>
</a><span style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px;"></span><div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<a href="http://%3Ccdn%3E/%3cchache%3e/%3cmachine" style="color: #005fa9; margin: 0px; padding: 0px; text-decoration: none;"></a><a href="https://www.blogger.com/null" name="OLE_LINK4" style="color: #22749b; margin: 0px; padding: 0px;"></a><a href="https://www.blogger.com/null" name="OLE_LINK3" style="color: #22749b; margin: 0px; padding: 0px;">七、TFS</a>解决方案概述</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<a href="http://code.taobao.org/p/tfs/src/" style="color: #005fa9; margin: 0px; padding: 0px; text-decoration: none;">http://code.taobao.org/p/tfs/src/</a></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
TFS(Taobao !FileSystem)是一个高可扩展、高可用、高性能、面向互联网服务的分布式文件系统,主要针对海量的非结构化数据,它构筑在普通的Linux机器 集群上,可为外部提供高可靠和高并发的存储访问。TFS为淘宝提供海量小文件存储,通常文件大小不超过1M,满足了淘宝对小文件存储的需求,被广泛地应用 在淘宝各项应用中。它采用了HA架构和平滑扩容,保证了整个文件系统的可用性和扩展性。同时扁平化的数据组织结构,可将文件名映射到文件的物理地址,简化 了文件的访问流程,一定程度上为TFS提供了良好的读写性能。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="376" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204303_252.gif" style="border: 0px; margin: 0px; padding: 0px;" width="651" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
TFS的块大小可以通过配置项来决定,通常使用的块大小为64M。TFS的设计目标是海量小文件的存储,所以每个块中会存储许多不同的小文 件。!DataServer进程会给Block中的每个文件分配一个ID(File ID,该ID在每个Block中唯一),并将每个文件在Block中的信息存放在和Block对应的Index文件中。这个Index文件一般都会全部 load在内存,除非出现!DataServer服务器内存和集群中所存放文件平均大小不匹配的情况。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
TFS中之所以可以使用namenode存放元数据信息的一个原因在于不像HDFS的元数据需要存放,filename与block id的映射以及block id与datanode的映射。在TFS中没有file的概念,只有block 的映射信息。所有的小文件被拼接成block。所以namenode中只需要存放<file block="" id="" length="" name="" offset="" style="margin: 0px; padding: 0px;">的映射以及<block datanode="" id="" style="margin: 0px; padding: 0px;">的映射。这样一来元数据信息就会减少很多,从而解决HDFS的namenode的瓶颈问题。</block></file></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
在TFS中,将大量的小文件(实际用户文件)合并成为一个大文件,这个大文件称为块(Block)。TFS以Block的方式组织文件的存储。每一 个Block在整个集群内拥有唯一的编号,这个编号是由NameServer进行分配的,而DataServer上实际存储了该Block。 在!NameServer节点中存储了所有的Block的信息,一个Block存储于多个!DataServer中以保证数据的冗余。对于数据读写请求, 均先由!NameServer选择合适的!DataServer节点返回给客户端,再在对应的!DataServer节点上进行数据操 作。!NameServer需要维护Block信息列表,以及Block与!DataServer之间的映射关系,其存储的元数据结构如下:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="331" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204304_120.gif" style="border: 0px; margin: 0px; padding: 0px;" width="477" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
八、一种提高云存储小文件效率的解决方案</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<a href="http://www.chinacloud.cn/show.aspx?&id=8105&cid=30" style="color: #005fa9; margin: 0px; padding: 0px; text-decoration: none;">http://www.chinacloud.cn/show.aspx?&id=8105&cid=30</a></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
(美国西北太平洋国家实验室2007年的一份研究报告表明,他们系统中有1 200万个文件,其中94%的文件小于64 MB,58%的小于64 kB。在一些具体的科研计算环境中,也存在大量的小文件,例如,在某些生物学计算中可能会产生3 000万个文件,而其平均大小只有190 kB。)</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="228" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204304_623.gif" style="border: 0px; margin: 0px; padding: 0px;" width="449" /></div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
系统为每个用户建立了3种队列:</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
序列文件队列(SequenceFile queue,SFQ),</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
序列文件操作队列(SequenceFile operation queue,SFOQ),</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
备用队列(Backup queue,BQ)。</div>
<div style="background-color: #fafafa; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
其中,SFQ用于小文件的合并,SFOQ用于对合并后小文件的操作,BQ用于操作的小文件数超过SFQ或SFOQ长度的情况。</div>
<img alt="HDFS小文件处理解决方案总结+facebook(HayStack) + 淘宝(TFS) " height="442" src="http://static.open-open.com/lib/uploadImg/20120301/20120301204304_30.gif" style="background-color: #fafafa; border: 0px; color: #404040; font-family: 'Microsoft YaHei', Verdana, sans-serif, SimSun; font-size: 14px; line-height: 25.2000007629395px; margin: 0px; padding: 0px;" width="399" /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<h1 class="post-title" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 42px; font-weight: 300; letter-spacing: -1px; line-height: 1; margin: 0px 0px 15px; padding: 0px;">
Hadoop小文件问题</h1>
</div>
<div>
http://www.gfzj.us/series/small-files/2015/01/06/hadoop-small-file.html</div>
<div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
海量小文件对于Hadoop来说是一个灾难。如果不可避免的要使用Hadoop处理小文件,此处提供一些方案。</div>
<h3 id="problems-with-small-files-and-hdfs" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
Problems with small files and HDFS</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
小文件是指那些文件大小比HDFS block size(默认64M)小很多的文件。每个小文件即使size很小,仍旧会占用一个block,不会多个小文件共用一个block。如果你在使用Hadoop时需要存储小文件,那么就意味着你可能有很多小文件,否则不会选择使用Hadoop。但是,问题就在于HDFS不能处理大量的文件。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
HDFS中每一个文件、目录或是block在namenode的内存中都对应一个对象,每个对象占用150个字节,因此,一千万个小文件,每个占用一个block,那么namenode就要消耗大约2G的内存。如果有更多的小文件,意味着namenode需要消耗更多的内存,而现有的计算机硬件可能会难以满足相应需求,且会导致集群难以扩展。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
另外,HDFS并不是为了有效地访问小文件而设计的:其初衷是为了流式访问大文件。如果访问大量小文件,需要执行大量的seeks操作,并需要不断地从一个datanode跳到另一个datanode,从而获取每个小文件,然而,上述每个操作都是低效的数据访问方式。</div>
<h3 id="problems-with-small-files-and-mapreduce" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
Problems with small files and MapReduce</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
Map tasks通常是以block为单位进行数据的处理。如果文件非常小且文件数量极大,那么每个map task处理的数据就非常少,且需要启动大量的map tasks,而记录每个map task信息(<strong>bookkeeping</strong>)也需要一定的开销。举个例子:一个是单独的1GB的文件,在HDFS中存储到16个64MB blocks;另一个是10000个100KB的小文件,大约共1GB。这10000个文件每个都用一个map task处理,那么处理这些文件所需的时间要比第一种情况慢上十倍甚至百倍。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
Hadoop提供了一些方法用于减少bookkeeping带来的开销:设置mapred.job.reuse.jvm.num.tasks属性,允许一个JVM同时执行多个map tasks,以这种重用task JVM的方式减少启动多个JVM的开销;使用MultiFileInputSplit,每个map task可处理多个blocks数据。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<strong>PS:</strong> bookkeeping是指在一个job的初始化阶段记录每个task的状态和进度。</div>
<h3 id="one-example" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
One Example</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
一个非常典型的小文件案例就是存储海量图片,每个图片是一个单独的小文件,这种情况就需要使用一个容器把图片进行分组打包存储。</div>
<h3 id="har-files" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
HAR files</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
为了缓解大量小文件带给namenode内存的压力,Hadoop 0.18.0引入了<a href="http://hadoop.apache.org/docs/r1.0.4/cn/hadoop_archives.html" style="color: #1756a9; text-decoration: none;" title="Hadoop Archives">Hadoop Archives</a>(HAR files),其本质就是在HDFS之上构建一个分层文件系统。通过执行<code style="background-color: #eeeeff; border-radius: 3px; border: 1px solid rgb(232, 232, 232); font-size: 15px; padding: 1px 5px;">hadoop archive</code>命令就可以创建一个HAR文件。在命令行下,用户可使用一个以<code style="background-color: #eeeeff; border-radius: 3px; border: 1px solid rgb(232, 232, 232); font-size: 15px; padding: 1px 5px;">har://</code>开头的URL就可以访问HAR文件中的小文件。使用HAR files可以减少HDFS中的文件数量。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
下图为HAR文件的文件结构,可以看出来访问一个指定的小文件需要访问两层索引文件才能获取小文件在HAR文件中的存储位置,因此,访问一个HAR文件的效率可能会比直接访问HDFS文件要低。对于一个MapReduce任务来说,如果使用HAR文件作为其输入,仍旧是其中每个小文件对应一个Map task,效率低下。所以,HAR files最好是用于文件归档。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<img alt="HAR File Layout" data-bd-imgshare-binded="1" src="http://www.gfzj.us/images/har.png" style="background: rgb(255, 255, 255); border: 0.5em solid rgb(255, 255, 255); box-shadow: rgba(0, 0, 0, 0.14902) 0px 1px 4px; box-sizing: border-box; display: block; margin: 20px auto; max-width: 100%; text-align: center; vertical-align: middle;" title="HAR File Layout" /></div>
<h3 id="sequence-files" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
Sequence Files</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
除了HAR files,另一种可选是SequenceFile,其核心是以文件名为key、文件内容为value组织小文件。回到之前提到的10000个100KB大小的文件,你可以编写程序将这些文件放到一个SequenceFile文件,然后就以数据流的方式处理这些文件,也可以使用MapReduce进行处理。一个SequenceFile是可分割的,所以MapReduce可将文件切分成块,每一块独立操作。不像HARs,SequenceFile支持压缩。在大多数情况下,以block为单位进行压缩是最好的选择,因为一个block包含多条记录,压缩作用在block之上,比Record压缩方式(一条一条记录进行压缩)的压缩比高。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
把已有的数据转存为SequenceFile比较慢。比起先写小文件,再将小文件写入SequenceFile,一个更好的选择是直接将数据写入一个SequenceFile文件,省去小文件作为中间媒介。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
下图为SequenceFile的文件结构。HAR files可以列出所有keys,但是SequenceFile是做不到的,因此,在访问时,只能从文件头顺序访问。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<img alt="SequenceFile File Layout" data-bd-imgshare-binded="1" src="http://www.gfzj.us/images/sequencefile.png" style="background: rgb(255, 255, 255); border: 0.5em solid rgb(255, 255, 255); box-shadow: rgba(0, 0, 0, 0.14902) 0px 1px 4px; box-sizing: border-box; display: block; margin: 20px auto; max-width: 100%; text-align: center; vertical-align: middle;" title="SequenceFile File Layout" /></div>
<h3 id="hbase" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
HBase</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
HBase也可用于存储小文件,前提是文件真的很小。</div>
<h3 id="section" style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 26px; font-weight: 300; margin: 0px 0px 15px; padding: 0px;">
个人总结</h3>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
对于海量小文件,该如何处理:</div>
<ol style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin: 0px 0px 15px 30px; padding: 0px;">
<li><div style="margin-bottom: 15px; padding: 0px;">
如果文件大小能保证在一个较小的范围内,使用HBase</div>
</li>
<li><div style="margin-bottom: 15px; padding: 0px;">
如果小文件的大小不能保证,考虑将文件直接写入HDFS,并在HBase中存储其实际地址</div>
</li>
<li><div style="margin-bottom: 15px; padding: 0px;">
如果小文件是每天都产生,那么可以考虑使用HAR files或者SequenceFile(或基于其的方式,如MapFile),并将源文件删除,在HBase中存储实际地址。</div>
</li>
</ol>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
如果用户需要通过HBase中的地址访问存储在HDFS中的小文件,那么就需要写相关服务来提供该功能了。</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<br /></div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<br /></div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<br /></div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<span style="background-color: white; color: #565656; font-family: 宋体, Arial; font-size: 12px; line-height: 9.60000038146973px;"> </span><a href="http://blog.chinaunix.net/uid-20577907-id-3989644.html" style="background-color: white; color: #19599b; font-family: 微软雅黑, 黑体, Verdana; font-size: 20px; text-decoration: none; word-wrap: break-word;">Hadoop小文件存储优化</a></div>
<div style="background-color: #fdfdfd; margin-bottom: 15px; padding: 0px;">
<span style="color: #5c5c5c; font-family: Helvetica Neue, Helvetica, Arial, Verdana, Hiragino Sans GB, Noto Sans CJK SC, Source Han Sans CN, STHeiti, Microsoft YaHei, WenQuanYi Micro Hei, SimSun, sans-serif;"><span style="font-size: 16px; line-height: 27.2000007629395px;">http://blog.chinaunix.net/uid-20577907-id-3989644.html</span></span></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<strong>一、概述</strong></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
首先明确概念,这里的小文件是指小于HDFS系统Block大小的文件(默认64M),如果使用HDFS存储大量的小文件,将会是一场灾难,这取决于HDFS的实现机制和框架结构,每一个存储在HDFS中的文件、目录和块映射为一个对象存储在NameNode服务器内存中,通常占用150个字节。如果有1千万个文件,就需要消耗大约3G的内存空间。如果是10亿个文件呢,简直不可想象。这里需要特别说明的是,<strong>每一个小于Block大小的文件,存储是实际占用的存储空间仍然是实际的文件大小,而不是整个block大小</strong>。</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
为解决小文件的存储Hadoop自身提供了两种机制来解决相关的问题,包括HAR和SequeueFile,这两种方式在某些方面解决了本层面的问题,单仍然存在着各自的不足。下文讲详细说明。</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<strong>二、Hadoop HAR</strong></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<a href="http://hadoop.apache.org/core/docs/current/hadoop_archives.html" style="color: navy; text-decoration: none;"> Hadoop Archives</a> (HAR files) ,这个特性从Hadoop 0.18.0版本就已经引入了,他可以将众多小文件打包成一个大文件进行存储,并且打包后原来的文件仍然可以通过Map-reduce进行操作,打包后的文件由索引和存储两大部分组成,索引部分记录了原有的目录结构和文件状态。其原理如下图所示:<img alt="" src="http://images.cnitblog.com/blog/496966/201303/08152825-49777ef19b134c7b9ca245545e1c5063.png" style="border: 0px; width: 813px;" /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<br /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<br /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
缺点:</div>
<ol style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; padding-left: 50px;">
<li style="list-style-type: decimal;">HAR 方式虽然能够实现NameNode内存空间的优化,但是他是一个人工干预的过程,同时他既不能够支持自动删除原小文件,也不支持追加操作,当有新文件进来以后,需要重新打包。</li>
<li style="list-style-type: decimal;">HAR files一旦创建就不能修改,要做增加和修改文件必须重新打包。事实上,这对那些写后便不能改的文件来说不是问题,因为它们可以定期成批归档,比如每日或每周。</li>
<li style="list-style-type: decimal;">HAR files目前还不支持文档压缩。</li>
</ol>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<strong>三、SequeuesFile</strong></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
Sequence file由一系列的二进制key/value组成,如果key为小文件名,value为文件内容,则可以将大批小文件合并成一个大文件。Hadoop-0.21.0版本开始中提供了SequenceFile,包括Writer,Reader和SequenceFileSorter类进行写,读和排序操作。该方案对于小文件的存取都比较自由,不限制用户和文件的多少,支持Append追加写入,支持三级文档压缩(不压缩、文件级、块级别)。其存储结构如下图所示:</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<img alt="" src="http://images.cnitblog.com/blog/496966/201303/08154404-6a467b2d478745428a0b59162834e26e.png" style="border: 0px;" /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
示例代码如下所示:</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
private static void writeTest(FileSystem fs, int count, int seed, Path file,<br /> CompressionType compressionType, CompressionCodec codec)<br /> throws IOException {<br /> fs.delete(file, true);<br /> LOG.info("creating " + count + " records with " + compressionType +<br /> " compression");</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
//指明压缩方式<br /> SequenceFile.Writer writer =<br /> SequenceFile.createWriter(fs, conf, file,<br /> RandomDatum.class, RandomDatum.class, compressionType, codec);<br /> RandomDatum.Generator generator = new RandomDatum.Generator(seed);<br /> for (int i = 0; i < count; i++) {<br /> generator.next();</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
//keyh<br /> RandomDatum key = generator.getKey();</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
//value<br /> RandomDatum value = generator.getValue();<br /> //追加写入<br /> writer.append(key, value);<br /> }<br /> writer.close();<br /> }</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
缺点:</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
目前为止只发现其Java版本API支持,未在其他开发接口中发现相关版本的实现,尤其是LibHDFS和thrift接口中,可能真是C++阵营狂热支持者的一个悲剧。</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<strong>四、Hbase</strong></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
如果你需要处理大量的小文件,并且依赖于特定的访问模式,可以采用其他的方式,比如Hbase。Hbase以MapFiles存储文件,并支持Map/Reduce格式流数据分析。对于大量小文件的处理,也不失为一种好的选择。</div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<br /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<br /></div>
<div style="background-color: white; font-family: Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 10px auto;">
<br /></div>
<h1 style="border-bottom-color: rgb(216, 217, 217); border-bottom-style: solid; border-bottom-width: 1px; font-family: 宋体; font-size: 22px; height: auto; line-height: 42px; margin: 0px; padding: 12px 0px 0px; text-align: center;">
Hadoop关于处理大量小文件的问题和解决方法</h1>
<div>
http://os.51cto.com/art/201310/413719.htm</div>
<div>
<br /></div>
<div>
<div class="brief bgF8F8F8" style="background: rgb(248, 248, 248); border-left-color: rgb(216, 217, 217); border-left-width: 1px; border-right-color: rgb(216, 217, 217); border-right-width: 1px; border-style: solid solid none; border-top-color: rgb(216, 217, 217); border-top-width: 1px; color: #333333; font-family: 宋体; font-size: 12px; line-height: 18px; margin-top: 10px; padding: 12px 18px 0px; width: 597px;">
<div class="brieftext" style="background: rgb(255, 255, 255); border: 1px dashed rgb(115, 160, 207); line-height: 26px; padding: 8px 12px;">
<div class="f14 green" style="background-color: transparent; color: #006600; font-size: 14px; padding: 0px;">
小文件指的是那些size比HDFS的block size(默认64M)小的多的文件。如果在HDFS中存储小文件,那么在HDFS中肯定会含有许许多多这样的小文件(不然就不会用hadoop了)。而HDFS的问题在于无法很有效的处理大量小文件。</div>
<div class="ad" style="background-color: transparent; padding: 0px; text-align: right;">
AD:<a href="http://mobile.51cto.com/mobile/mdsa04/" style="color: red; margin: 5px; text-decoration: none;" target="_blank">51CTO移动APP安全沙龙!马上要爆满,手慢没座位!</a></div>
</div>
</div>
<div class="content bgF8F8F8 f14" style="background: rgb(248, 248, 248); border-bottom-color: rgb(216, 217, 217); border-bottom-width: 1px; border-left-color: rgb(216, 217, 217); border-left-width: 1px; border-right-color: rgb(216, 217, 217); border-right-width: 1px; border-style: none solid solid; color: #333333; font-family: 宋体; font-size: 14px; line-height: 28px; margin-bottom: 10px; min-height: 500px; overflow: hidden; padding: 12px 18px 0px; position: relative;">
<div id="content">
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
小文件指的是那些size比HDFS的block size(默认64M)小的多的文件。如果在HDFS中存储小文件,那么在HDFS中肯定会含有许许多多这样的小文件(不然就不会用hadoop了)。而HDFS的问题在于无法很有效的处理大量小文件。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
任何一个文件,目录和block,在HDFS中都会被表示为一个object存储在namenode的内存中,没一个object占用150 bytes的内存空间。所以,如果有10million个文件,没一个文件对应一个block,那么就将要消耗namenode 3G的内存来保存这些block的信息。如果规模再大一些,那么将会超出现阶段计算机硬件所能满足的极限。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
不仅如此,HDFS并不是为了有效的处理大量小文件而存在的。它主要是为了流式的访问大文件而设计的。对小文件的读取通常会造成大量从datanode到datanode的seeks和hopping来retrieve文件,而这样是非常的低效的一种访问方式。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
<strong style="text-align: center;">大量小文件在mapreduce中的问题</strong></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
Map tasks通常是每次处理一个block的input(默认使用FileInputFormat)。如果文件非常的小,并且拥有大量的这种小文件,那么每一个map task都仅仅处理了非常小的input数据,并且会产生大量的map tasks,每一个map task都会消耗一定量的bookkeeping的资源。比较一个1GB的文件,默认block size为64M,和1Gb的文件,没一个文件100KB,那么后者没一个小文件使用一个map task,那么job的时间将会十倍甚至百倍慢于前者。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
hadoop中有一些特性可以用来减轻这种问题:可以在一个JVM中允许task reuse,以支持在一个JVM中运行多个map task,以此来减少一些JVM的启动消耗(通过设置mapred.job.reuse.jvm.num.tasks属性,默认为1,-1为无限制)。另一种方法为使用MultiFileInputSplit,它可以使得一个map中能够处理多个split。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
<strong style="text-align: center;">为什么会产生大量的小文件?</strong></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
<span style="color: navy;">至少有两种情况下会产生大量的小文件</span></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
1.这些小文件都是一个大的逻辑文件的pieces。由于HDFS仅仅在不久前才刚刚支持对文件的append,因此以前用来向unbounde files(例如log文件)添加内容的方式都是通过将这些数据用许多chunks的方式写入HDFS中。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
2.文件本身就是很小。例如许许多多的小图片文件。每一个图片都是一个独立的文件。并且没有一种很有效的方法来将这些文件合并为一个大的文件</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
这两种情况需要有不同的解决方式。对于第一种情况,文件是由许许多多的records组成的,那么可以通过件邪行的调用HDFS的sync()方法(和append方法结合使用)来解决。或者,可以通过些一个程序来专门合并这些小文件(see Nathan Marz’s post about a tool called the Consolidator which does exactly this)。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
对于第二种情况,就需要某种形式的容器来通过某种方式来group这些file。hadoop提供了一些选择:</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
<strong style="text-align: center;">HAR files</strong></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
Hadoop Archives (HAR files)是在0.18.0版本中引入的,它的出现就是为了缓解大量小文件消耗namenode内存的问题。HAR文件是通过在HDFS上构建一个层次化的文件系统来工作。一个HAR文件是通过hadoop的archive命令来创建,而这个命令实 际上也是运行了一个MapReduce任务来将小文件打包成HAR。对于client端来说,使用HAR文件没有任何影响。所有的原始文件都 visible && accessible(using har://URL)。但在HDFS端它内部的文件数减少了。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-align: center; text-indent: 28px;">
<a href="http://s2.51cto.com/wyfs01/M02/31/1E/wKioOVJnOYXDngDAAAAs4Vh_ex8959.jpg" style="color: #004276;" target="_blank"><img alt="" class="fit-image" src="http://s2.51cto.com/wyfs01/M02/31/1E/wKioOVJnOYXDngDAAAAs4Vh_ex8959.jpg" style="border: 0px;" /></a></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
通过HAR来读取一个文件并不会比直接从HDFS中读取文件高效,而且实际上可能还会稍微低效一点,因为对每一个HAR文件的访问都需要完成两层index文件的读取和文件本身数据的读取(见上图)。并且尽管HAR文件可以被用来作为MapReduce job的input,但是并没有特殊的方法来使maps将HAR文件中打包的文件当作一个HDFS文件处理。可以考虑通过创建一种input format,利用HAR文件的优势来提高MapReduce的效率,但是目前还没有人作这种input format。需要注意的是:MultiFileInputSplit,即使在HADOOP-4565的改进(choose files in a split that are node local),但始终还是需要seek per small file。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
<strong style="text-align: center;">Sequence Files</strong></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
通常对于“the small files problem”的回应会是:使用SequenceFile。这种方法是说,使用filename作为key,并且file contents作为value。实践中这种方式非常管用。回到10000个100KB的文件,可以写一个程序来将这些小文件写入到一个单独的SequenceFile中去,然后就可以在一个streaming fashion(directly or using mapreduce)中来使用这个sequenceFile。不仅如此,SequenceFiles也是splittable的,所以mapreduce可以break them into chunks,并且分别的被独立的处理。和HAR不同的是,这种方式还支持压缩。block的压缩在许多情况下都是最好的选择,因为它将多个records压缩到一起,而不是一个record一个压缩。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
将已有的许多小文件转换成一个SequenceFiles可能会比较慢。但是,完全有可能通过并行的方式来创建一个一系列的SequenceFiles。(Stuart Sierra has written a very useful post about converting a tar file into a SequenceFile—tools like this are very useful)。更进一步,如果有可能最好设计自己的数据pipeline来将数据直接写入一个SequenceFile。</div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-align: center; text-indent: 28px;">
<a href="http://s5.51cto.com/wyfs01/M00/31/1C/wKioJlJnOYXBhEI7AABLh8StylU307.jpg" style="color: #004276;" target="_blank"><img alt="" class="fit-image" src="http://s5.51cto.com/wyfs01/M00/31/1C/wKioJlJnOYXBhEI7AABLh8StylU307.jpg" style="border: 0px;" width="498" /></a></div>
<div style="background-color: transparent; margin-bottom: 10px; margin-top: 10px; padding: 0px; text-indent: 28px;">
【编辑推荐】</div>
<div>
<ol style="list-style: none outside none; margin: 0px 0px 0px 55px; padding: 0px; word-break: normal; word-wrap: break-word;">
<li style="list-style: decimal outside none; margin: 0px; padding: 0px; word-break: normal; word-wrap: break-word;"><a href="http://os.51cto.com/art/201304/390393.htm" style="color: #004276;" target="_blank">Apache Hadoop 为什么如此风靡?</a></li>
<li style="list-style: decimal outside none; margin: 0px; padding: 0px; word-break: normal; word-wrap: break-word;"><a href="http://os.51cto.com/art/201304/390896.htm" style="color: #004276;" target="_blank">小议Hadoop HDFS Balancer</a></li>
<li style="list-style: decimal outside none; margin: 0px; padding: 0px; word-break: normal; word-wrap: break-word;"><a href="http://os.51cto.com/art/201305/396145.htm" style="color: #004276;" target="_blank">技术小白:Hadoop 到底是啥?</a></li>
<li style="list-style: decimal outside none; margin: 0px; padding: 0px; word-break: normal; word-wrap: break-word;"><a href="http://os.51cto.com/art/201308/408064.htm" style="color: #004276;" target="_blank">关于Hadoop的六大误解</a></li>
<li style="list-style: decimal outside none; margin: 0px; padding: 0px; word-break: normal; word-wrap: break-word;">用Hadoop,还是不用Hadoop?</li>
</ol>
</div>
</div>
</div>
</div>
<div style="background-color: #fdfdfd; color: #5c5c5c; font-family: 'Helvetica Neue', Helvetica, Arial, Verdana, 'Hiragino Sans GB', 'Noto Sans CJK SC', 'Source Han Sans CN', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, sans-serif; font-size: 16px; line-height: 27.2000007629395px; margin-bottom: 15px; padding: 0px;">
<br /></div>
</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-67640338213560889802014-05-20T19:47:00.002-07:002014-05-20T19:47:58.724-07:00網路服務第三方認證OAUTH (http://oauth.net/) - An open protocol to allow secure authorization in a simple and standard method from web, mobile and desktop applications.<br />
<br />
Yahoo OAuth (https://developer.yahoo.com/oauth/guide/oauth-guide.html)<br />
<span style="color: #222222; font-family: 'Open Sans', sans-serif; line-height: 24px;"><br /></span>
<span style="color: #222222; font-family: 'Open Sans', sans-serif; line-height: 24px;">一步一步搭建OAuth認證伺服器 </span><br />
<span style="color: #222222; font-family: Open Sans, sans-serif;"><span style="line-height: 24px;">-- http://fecbob.pixnet.net/blog/post/38124119-%5Bphp%5D-%E4%B8%80%E6%AD%A5%E4%B8%80%E6%AD%A5%E6%90%AD%E5%BB%BAoauth%E8%AA%8D%E8%AD%89%E4%BC%BA%E6%9C%8D%E5%99%A8</span></span><br />
<span style="color: #222222; font-family: 'Open Sans', sans-serif; line-height: 24px;"><br /></span>
<span style="color: #222222; font-family: 'Open Sans', sans-serif; line-height: 24px;">Authorization for Google Services</span><br />
<span style="color: #222222; font-family: 'Open Sans', sans-serif; line-height: 24px;">-https://developers.google.com/apps-script/guides/services/authorization</span><br />
<br />
<h3 class="r" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: medium; font-weight: normal; margin: 0px; overflow: hidden; padding: 0px; text-overflow: ellipsis; white-space: nowrap;">
<span style="color: #dd4b39;">OAuth</span> - Open Authorization 開放式授權<span style="color: #dd4b39;">協定</span></h3>
<div>
<span style="color: #dd4b39;">- </span><span style="color: #dd4b39;">http://blog.masterstudio101.com/2013/04/24/OAuth%20-%20Open%20Authorization%20%E9%96%8B%E6%94%BE%E5%BC%8F%E6%8E%88%E6%AC%8A%E5%8D%94%E5%AE%9A</span></div>
<div>
<span style="color: #dd4b39;"><br /></span></div>
<div>
<span style="color: #dd4b39;"><br /></span></div>
<div>
<span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">詳細RFC6749內容請參考 </span><a href="http://tools.ietf.org/html/rfc6749" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://tools.ietf.org/html/rfc6749</a><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><strong style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">下面列出知名網站的OAuth 2.0的網址(URLs)</strong><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Google</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://accounts.google.com/o/oauth2/auth</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://accounts.google.com/o/oauth2/token</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://www.googleapis.com/oauth2/v1</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Facebook</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://www.facebook.com/dialog/oauth</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://graph.facebook.com/oauth/access_token</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://graph.facebook.com</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">StackExchange (https://api.stackexchange.com/docs/authentication)</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://stackexchange.com/oauth</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://stackexchange.com/oauth/access_token</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://api.stackexchange.com/2.0</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Github</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://github.com/login/oauth/authorize</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://github.com/login/oauth/access_token</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> https://api.github.com</span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">另外補充, OAuth與OpenID (Google, PayPal, VeriSign)最大不同在於, OAuth是進行資源的授權(Authorization), 而OpenID是進行身份的驗證(Authentication), 兩者並不衝突更可予以整合應用, 有關OpenID細節容study後再分享. </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><strong style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Reference 參考資料:</strong><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">OAuth協定與運作流程 </span><a href="http://cire.pixnet.net/blog/post/30810748-%E6%BC%AB%E8%AB%87oauth%E8%AA%8D%E8%AD%89%E5%8D%94%E5%AE%9A%E8%88%87%E9%81%8B%E4%BD%9C%E6%B5%81%E7%A8%8B" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://cire.pixnet.net/blog/post/308107...</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">OAuth和OpenID </span><a href="http://sinkwung.blogspot.tw/2012/04/oauth-openid.html" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://sinkwung.blogspot.tw/...</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">OAuth 參數說明 </span><a href="http://www.dotblogs.com.tw/regionbbs/archive/2011/04/21/oauth.parameters.aspx" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://www.dotblogs.com.tw/regionbbs/...</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><strong style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">Resources 資源:</strong><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">OAuth 官方網站 http://oauth.net/ </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">OAuth 2.0官方網站 http://oauth.net/2 </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">RFC6749 </span><a href="http://tools.ietf.org/html/rfc6749" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://tools.ietf.org/html/rfc6749</a><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Yahoo Developer Network </span><a href="http://developer.yahoo.com/" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://developer.yahoo.com</a><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">StackExchange OAuth </span><a href="https://api.stackexchange.com/docs/authentication" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">https://api.stackexchange.com/docs/authentication</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Using OAuth 2.0 to access Google API</span><a href="https://developers.google.com/accounts/docs/OAuth2" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">https://developers.google.com/accounts/docs/OAuth2</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Facebook Authentication</span><a href="https://developers.facebook.com/docs/authentication" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">https://developers.facebook.com/docs/authentication</a><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Windows Live oAuth 2.0 </span><a href="http://msdn.microsoft.com/en-us/library/live/hh243647.aspx" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://msdn.microsoft.com/en-us/library/live/hh243647.aspx</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span><br style="background-color: white; border: 0px; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;" /><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;">Wiki OAuth </span><a href="http://zh.wikipedia.org/wiki/OAuth" style="background-color: white; border: 0px; color: #03a7eb; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://zh.wikipedia.org/wiki/OAuth</a><span style="background-color: white; color: #181818; font-family: Verdana, Arial; font-size: 19px; line-height: 27.900001525878906px;"> </span></div>
<ul class="article-head" style="background-color: white; list-style: none; margin: 10px 0px; padding: 5px;">
<li class="title" data-hot-link="//www.pixnet.net/blog/articles/category/0" data-site-category="數位生活" id="article-30810748" style="margin: 0px; padding: 0px;"><div style="margin: 10px 0px; padding: 0px;">
<span style="color: #222222; font-family: 'ms ui gothic', verdana, arial, sans-serif; font-size: 13pt; font-weight: normal; line-height: 21.600000381469727px;">漫談OAuth認證協定與運作流程 </span></div>
</li>
<li class="title" data-hot-link="//www.pixnet.net/blog/articles/category/0" data-site-category="數位生活" id="article-30810748" style="margin: 0px; padding: 0px;"><div style="margin: 10px 0px; padding: 0px;">
<span style="color: #222222; font-family: ms ui gothic, verdana, arial, sans-serif;"><span style="font-size: 17px; line-height: 21.600000381469727px;">- http://cire.pixnet.net/blog/post/30810748-%E6%BC%AB%E8%AB%87oauth%E8%AA%8D%E8%AD%89%E5%8D%94%E5%AE%9A%E8%88%87%E9%81%8B%E4%BD%9C%E6%B5%81%E7%A8%8B</span></span></div>
</li>
</ul>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-39709087299485075672014-05-05T23:47:00.000-07:002014-07-08T17:49:57.020-07:00ubuntu如何備份和恢復source: http://www.itkee.com/os/detail-1476.html<br />
<br />
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在 使用Ubuntu之前,相信很多人都有過使用Windows系統的經歷。如果你備份過Windows系統,那麼你一定記憶猶新:首先需要找到一個備份工 具(通常都是私有軟件),然後重啓電腦進入備份工具提供的軟件環境,在這裏備份或者恢復Windows系統。Norton Ghost是備份Windows系統時經常使用的備份工具。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在備份Windows系統的時候你可能想過,我能不能把整個C盤都放到一個ZIP文件裏去呢。這在Windows下是不可能的,因爲在Windows中有很多文件在它們運行時是不允許拷貝或覆蓋的,因此你需要專門的備份工具對Windows系統進行特殊處理。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
和 備份Windows系統不同,如果你要備份Ubuntu系統(或者其它任何Linux系統),你不再需要像Ghost這類備份工具。事實上,Ghost 這類備份工具對於Linux文件系統的支持很糟糕,例如一些Ghost版本只能完善地支持Ext2文件系統,如果你用它來備份Ext3文件系統,你可能會 丟失一些寶貴的數據。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
1. 備份系統</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
我該如何備份我的Ubuntu系統呢?很簡單,就像你備份或壓縮其它東西一樣,使用TAR。和Windows不同,Linux不會限制root訪問任何東西,你可以把分區上的所有東西都扔到一個TAR文件裏去!</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
首先成爲root用戶:<br />
$ sudo su</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
然後進入文件系統的根目錄(當然,如果你不想備份整個文件系統,你也可以進入你想要備份的目錄,包括遠程目錄或者移動硬盤上的目錄):<br />
# cd /</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
下面是我用來備份系統的完整命令:<br />
# tar cvpzf backup.tgz Cexclude=/proc Cexclude=/lost+found Cexclude=/backup.tgz Cexclude=/mnt Cexclude=/sys /</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
讓我們來簡單看一下這個命令:</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
“tar”當然就是我們備份系統所使用的程序了。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
“cvpfz”是tar的選項,意思是“創建檔案文件”、“保持權限”(保留所有東西原來的權限)、“使用gzip來減小文件尺寸”。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
“backup.gz”是我們將要得到的檔案文件的文件名。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
“/”是我們要備份的目錄,在這裏是整個文件系統。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在 檔案文件名“backup.gz”和要備份的目錄名“/”之間給出了備份時必須排除在外的目錄。有些目錄是無用的,例如“/proc”、“/lost+ found”、“/sys”。當然,“backup.gz”這個檔案文件本身必須排除在外,否則你可能會得到一些超出常理的結果。如果不把“/mnt”排 除在外,那麼掛載在“/mnt”上的其它分區也會被備份。另外需要確認一下“/media”上沒有掛載任何東西(例如光盤、移動硬盤),如果有掛載東西, 必須把“/media”也排除在外。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
有人可能會建議你把“/dev”目錄排除在外,但是我認爲這樣做很不妥,具體原因這裏就不討論了。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
執行備份命令之前請再確認一下你所鍵入的命令是不是你想要的。執行備份命令可能需要一段不短的時間。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
備份完成後,在文件系統的根目錄將生成一個名爲“backup.tgz”的文件,它的尺寸有可能非常大。現在你可以把它燒錄到DVD上或者放到你認爲安全的地方去。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在備份命令結束時你可能會看到這樣一個提示:’tar: Error exit delayed from previous errors’,多數情況下你可以忽略它。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
你還可以用Bzip2來壓縮文件,Bzip2比gzip的壓縮率高,但是速度慢一些。如果壓縮率對你來說很重要,那麼你應該使用Bzip2,用“j”代替命令中的“z”,並且給檔案文件一個正確的擴展名“bz2”。完整的命令如下:<br />
# tar cvpjf backup.tar.bz2 Cexclude=/proc Cexclude=/lost+found Cexclude=/backup.tar.bz2 Cexclude=/mnt Cexclude=/sys /</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
2. 恢復系統</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在進行恢復系統的操作時一定要小心!如果你不清楚自己在做什麼,那麼你有可能把重要的數據弄丟,請務必小心!</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
接着上面的例子。切換到root用戶,並把文件“backup.tgz”拷貝到分區的根目錄下。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
在 Linux中有一件很美妙的事情,就是你可以在一個運行的系統中恢復系統,而不需要用boot-cd來專門引導。當然,如果你的系統已經掛掉不能啓動了, 你可以用Live CD來啓動,效果是一樣的。你還可以用一個命令把Linux系統中的所有文件幹掉,當然在這裏我不打算給出這個命令!</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
使用下面的命令來恢復系統:<br />
# tar xvpfz backup.tgz -C /</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
如果你的檔案文件是使用Bzip2壓縮的,應該用:<br />
# tar xvpfj backup.tar.bz2 -C /</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
注意:上面的命令會用檔案文件中的文件覆蓋分區上的所有文件。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
執行恢覆命令之前請再確認一下你所鍵入的命令是不是你想要的,執行恢覆命令可能需要一段不短的時間。</div>
<div style="background-color: #fafafa; color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px; margin-bottom: 16px; padding: 0px; text-align: justify;">
恢覆命令結束時,你的工作還沒完成,別忘了重新創建那些在備份時被排除在外的目錄:<br />
# mkdir proc<br />
# mkdir lost+found<br />
# mkdir mnt<br />
# mkdir sys<br />
等等</div>
<div style="background-color: #fafafa; margin-bottom: 16px; padding: 0px; text-align: justify;">
<div style="color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px;">
當你重啓電腦,你會發現一切東西恢復到你創建備份時的樣子了!</div>
<div style="color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px;">
<br /></div>
<div style="color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体; font-size: 14px; line-height: 18px;">
<br /></div>
<span style="color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体;"><span style="font-size: 14px; line-height: 18px;">http://blog.riaproject.com/server-setting/1305/%E7%B0%A1%E6%98%93%E5%82%99%E4%BB%BDubuntu%E7%B3%BB%E7%B5%B1%E7%9A%84%E6%96%B9%E6%B3%95.html</span></span><br />
<span style="color: #333333; font-family: arial, helvetica, sans-serif, verdana, 宋体;"><span style="font-size: 14px; line-height: 18px;"><br /></span></span>
<h2 style="background-color: white; border: 0px; clear: both; color: black; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin: 0px 0px 0.8125em; outline: 0px; padding: 0px; text-align: start; vertical-align: baseline;">
方法2: 用Sbackup軟體來做</h2>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; text-align: start; vertical-align: baseline;">
用GUI介面是對指令模式非常排斥,或是懶人所設計的,<br />其實所做的工作跟方法1是相同的,<br />只是這次多了UI介面給你用,點一點按一按就可以了。<br /><img alt="sbackup" class="alignnone size-full wp-image-1329" height="298" src="http://blog.riaproject.com/wp-content/uploads/2009/07/sbackup01.png" style="border: 1px solid rgb(221, 221, 221); height: auto; margin-top: 0.4em; max-width: 97.5%; padding: 6px; width: auto;" title="sbackup" width="400" /></div>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; text-align: start; vertical-align: baseline;">
<br /></div>
</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com1tag:blogger.com,1999:blog-7057734095102577753.post-37893200733296832952014-03-30T20:05:00.000-07:002014-03-30T20:05:18.288-07:00防火牆技術<h1 style="font-family: 標楷體;">
穿越防火牆技術 </h1>
<div>
1. (穿越防火牆技術) http://www.cs.nccu.edu.tw/~lien/Writing/NGN/firewall.htm</div>
<div>
2. Libnice (http://nice.freedesktop.org/wiki/)</div>
<div>
3. PJNATH - Open Source ICE, STUN, and TURN Library (http://www.pjsip.org/pjnath/docs/html/index.htm)</div>
<div>
4. The TCP/IP guide (http://www.tcpipguide.com/free/t_IPNetworkAddressTranslationNATProtocol.htm)</div>
<div>
5.NAT Traversal and Peer to Peer (http://lab.howie.tw/2012/07/Peer-to-Peer-and-NAT-Traversal.html)<br />
6.<br />
<br />
<br />
<br />
Code Reference<br />
1. Google libjingle (https://developers.google.com/talk/libjingle/?hl=zh-TW&csw=1)<br />
2. STUNT (http://nutss.gforge.cis.cornell.edu/jstunt-examples.php)<br />
3. PJNATH (http://www.pjsip.org/pjnath/docs/html/index.htm)</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-15156995764156018102014-03-30T20:04:00.001-07:002014-03-30T20:04:56.612-07:00Difference Between NAT and ProxySource: http://www.differencebetween.com/difference-between-nat-and-vs-proxy/<br />
<br />
<div class="txtcolor" style="border: 0px; color: #2266bb; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">NAT vs Proxy </strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; vertical-align: baseline;">
</div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
Network Address Translation (NAT) is the process that modifies the IP address in a header of an IP packet, while it is travelling through a routing device. NAT allows one set of IP addresses to be used for traffic within a LAN (Local Area Network) and another set of IP addresses for outside traffic. One to one transformation of IP addresses are provided by the simplest form of NAT. Proxy (proxy server) is a server that is located between a client (who is looking for a resource) and some other server and acts as a mediator. The client requesting the resource connects to the proxy server and the proxy evaluates the request based on its filtering rules.</div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><br /></strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">What is NAT?</strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
NAT modifies IP address in a header of an IP packet, while it is travelling through a routing device. NAT allows one set of IP addresses to be used for traffic within a LAN and another set of IP addresses for outside traffic. One to one transformation of IP addresses are provided by the simplest form of NAT. NAT has several advantages. It improves the security of a LAN since it provides the option to hide internal IP addresses. Furthermore, as the IP addresses are only used internally, it will not cause any conflicts with IP addresses used in other organizations. Also, using a single internet connection for all the computers in a LAN is made possible by NAT. NAT works with the use of a NAT box, which is situated in the interface where the LAN is connected to the internet. It contains a set of valid IP addresses and it is responsible for performing the IP address translations.</div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><br /></strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">What is a Proxy?</strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
Proxy is a server that is located between a client (that is looking for a resource) and some other server and acts as a mediator. The client requesting the resource connects to the proxy server and the proxy evaluates the request based on its filtering rules. If the request is validated, proxy connects to the server and provides the requested resource to the client. On the other hand, proxy may satisfy the client’s request without going to the specified server. For this, the proxy uses a cache and any of the subsequent requests for the same resource are satisfied without contacting the specified server. Due to this, proxies can improve the performance greatly. Furthermore, proxies can be used to filter requests and prevent accessing some web sites.</div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><br /></strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
<strong style="border: 0px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">What is the difference between NAT and Proxy?</strong></div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; outline: 0px; padding: 0px; text-align: justify; vertical-align: baseline;">
NAT modifies IP address in a header of an IP packet, while it is travelling through a routing device and allows to use a different set of IP addresses for traffic within a LAN than the set of IP addresses for outside traffic, while a proxy is a server that is located between a client and some other server and acts as a mediator. NAT does not need any special application software to operate, whereas applications behind a proxy server must support proxy services and should be configured to use the proxy server.</div>
<div style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 0px; outline: 0px; padding: 0px; text-align: center; vertical-align: baseline;">
<div style="border: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">
</div>
</div>
<span style="border: 0px; font-family: Arial, Helvetica, sans-serif; font-size: 13px; line-height: 13px; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"><br /><br />Read more: <a href="http://www.differencebetween.com/difference-between-nat-and-vs-proxy/#ixzz2xVQ3TG5i" style="border: 0px; color: #003399; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;">http://www.differencebetween.com/difference-between-nat-and-vs-proxy/#ixzz2xVQ3TG5i</a></span>Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-80583477555090622972014-01-27T21:02:00.001-08:002014-01-27T21:02:32.182-08:00Ubuntu 遠端桌面畫面問題Source:http://blog.snowtec.org/2009/05/ubuntu-904-vnc-problem/<br />
<br />
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
Ubuntu內建了遠端桌面功能 (也就是VNC Server),在「系統→偏好設定→遠端桌面」可以進行開啟與其他設定。</div>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
不過最近我遇到了一個相當奇怪的Bug…連線成功之後,系統卻<strong style="border: 0px; font-family: inherit; font-style: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">不會將畫面更新</strong>,總是停留在第一個畫格。雖然動作事件都有送到被控端,但在看不到螢幕反應的情況下使用上實在是有點…不,是十分困難啊!遇到這種事情當然我們還是要求助於Google大神,然後很幸運的我得到了解答。</div>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
<span id="more-635" style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;"></span>這個Bug的發生時機是,你使用nVidia的顯示卡,然後也啟用了Compiz (也就是桌面特效)。Xwindow有個功能叫XDamage,在做VNC連線時可以只傳送桌面上改變的部份的畫面 (例如,你移動的視窗),進而減少使用網路傳輸量。但不知為什麼,遇到受限制的nVidia驅動程式加上Compiz,這個功能就會壞掉,變成client一旦連上之後畫面就不會更新,永遠卡在第一個frame。<br />在 <a href="https://bugs.launchpad.net/ubuntu/+source/compiz/+bug/353126" style="border: 0px; color: #084e93; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;" target="_blank">Launchpad</a> 已經有網友提出這個bug,也已經有解決辦法了。</div>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
解決方案有:</div>
<ol style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; list-style-image: initial; list-style-position: initial; margin: 0px 0px 1.625em 2.5em; outline: 0px; padding: 0px; vertical-align: baseline;">
<li style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">不使用VNC。 (這…)</li>
<li style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">完全關閉桌面特效。<br /><a href="http://p9.p.pixnet.net/albums/userpics/9/8/258198/4a2286dddb5d0.jpg" rel="lightbox[vnc]" style="border: 0px; color: #084e93; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;" title="ubuntu桌面特效"><img alt="ubuntu桌面特效" src="http://p9.p.pixnet.net/albums/userpics/9/8/258198/4a2286dddb5d0.jpg" style="border: 0px; max-width: 100%;" title="ubuntu桌面特效" width="300" /></a></li>
<li style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">使用別的VNC軟體,例如:x11vnc:<br /><blockquote style="border: 0px; font-family: Georgia, 'Bitstream Charter', serif; font-style: italic; margin: 0px 3em; outline: 0px; padding: 0px; quotes: ''; vertical-align: baseline;">
<div style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
<span style="border: 0px; color: grey; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">$</span> sudo apt-get install x11vnc<br /><span style="border: 0px; color: grey; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">$</span> x11vnc –noxdamage –passwd 密碼 -forever</div>
</blockquote>
</li>
<li style="border: 0px; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">安裝修正包:<br />Launchpad上已經放出了修正過後的Vino (內建的VNC Server),到 <a href="https://launchpad.net/~wendell/+archive/ppa" style="border: 0px; color: #084e93; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;" target="_blank">這裡</a> 可以下載更新過後的版本。<br />安裝完畢之後,按「Alt+F2」執行<span style="border: 0px; color: blue; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">「gconf-editor」</span>,把鍵值<br /><span style="border: 0px; color: red; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; vertical-align: baseline;">「/desktop/gnome/remote_access/disable_xdamage」</span>修改為true (勾起來的意思),再到選單中啟動遠端桌面。<br /><a href="http://p9.p.pixnet.net/albums/userpics/9/8/258198/4a228bed37b9c.jpg" rel="lightbox[vnc]" style="border: 0px; color: #084e93; font-family: inherit; font-style: inherit; font-weight: inherit; margin: 0px; outline: 0px; padding: 0px; text-decoration: none; vertical-align: baseline;" title="gconfeditor遠端連線修改"><img alt="gconfeditor遠端連線修改" src="http://p9.p.pixnet.net/albums/userpics/9/8/258198/4a228bed37b9c.jpg" style="border: 0px; max-width: 100%;" title="gconfeditor遠端連線修改" width="450" /></a></li>
</ol>
<div style="background-color: white; border: 0px; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; line-height: 24.375px; margin-bottom: 1.625em; outline: 0px; padding: 0px; vertical-align: baseline;">
雖然說這樣會影響連線品質,不過目前似乎是沒有更好的解法,我也不敢保證我的是最好最恰當的解法,所以如果有什麼意見的話也歡迎留言跟我說。<br />到這裡你的VNC應該就可以順利、快樂、至少不會停格的連線了。</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-23275913219419957942014-01-05T22:14:00.000-08:002014-01-05T22:14:31.194-08:003C應用<h1 class="headline" style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px; margin: 0px 0px 10px; padding: 0px;">
<span style="font-weight: normal;">1. <span style="font-size: 24.799999237060547px; line-height: 32.79999923706055px;">你的手機就是最高檔的行車記錄器</span></span></h1>
<div>
2. <span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">只是手寫何必買 Wacom?手機免費變身無線手寫板!</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">3. </span><span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">將#可穿戴設備#與汽車駕駛結合起來,現代汽車正開發一款#Google Glass#應用來操控汽車</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">4. </span><span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">HTC Mini+ – 超越配件等級的多功能智慧小手機</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">5. </span><a data-rapid_p="1" href="http://www.kocpc.com.tw/archives/4375" rel="nofollow" style="background-color: white; color: #333333; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 16px; line-height: 29.600000381469727px; text-decoration: none;">HTC Fetch 藍牙定位協尋器 – 讓你不再弄丟鑰匙或手機!</a></div>
<div>
6. <a data-rapid_p="2" href="http://www.kocpc.com.tw/archives/4341" rel="nofollow" style="background-color: white; color: #333333; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 16px; line-height: 29.600000381469727px; text-decoration: none;">HTC Power Flip Case – 不僅是保護殼!具行動電源功能的手機背夾</a></div>
<div>
7. <span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">輕便旅行隨身好物 i-FlashDrive雙頭龍Android特別版</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">8. </span><span style="color: #333333; font-size: 20px;">Smart Sign Language Interpreter智慧型手語翻譯 聽障人士代言人</span></div>
<div>
<span style="color: #333333; font-size: 20px;">9. </span><span style="background-color: white; font-family: Arial, Helvetica, sans-serif; font-size: 23.636363983154297px; line-height: 29.332386016845703px;">2014 CES 大預測:別管 Google Glass 了,穿戴式科技大舉入侵健康照護和健身領域!</span></div>
<div>
<span style="background-color: white; font-family: Arial, Helvetica, sans-serif; font-size: 23.636363983154297px; line-height: 29.332386016845703px;">10. </span><span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">Google再次聚焦家庭能源使用的追踪,正在測試連接到網絡的智能恆溫器</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">11. </span><span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;">想讓傳統家電變智能?只需配一個智能插座就行!Plugaway 推智能插座及智能LED燈</span></div>
<div>
<span style="background-color: white; font-family: arial, STHeiti, pmingliu, sans-serif; font-size: 24.799999237060547px; line-height: 32.79999923706055px;"><br /></span></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-62244236661063502482013-12-23T17:35:00.001-08:002013-12-23T17:35:40.251-08:00SVN 教學1. <strong style="color: #49b3c1; font-family: Verdana, Arial, Helvetica, �啁敦��; font-size: 16px; line-height: 24px; text-align: justify;">Subversion(SVN)概念與工具介紹</strong><br />
<a href="http://www.cc.ntu.edu.tw/chinese/epaper/0020/20120320_2002.html">http://www.cc.ntu.edu.tw/chinese/epaper/0020/20120320_2002.html</a><br />
<br />
2. <span style="color: #333333; font-family: Verdana, Geneva, Lucida; font-weight: bold;">SVN 入門安裝設定教學</span><br />
<a href="http://ithelp.ithome.com.tw/question/10078555?tag=rt.rq">http://ithelp.ithome.com.tw/question/10078555?tag=rt.rq</a><br />
<br />
3. <span style="color: #4b4b4b; font-family: 微軟正黑體, Oswald, sans-serif; font-size: 1.3rem;">Subversion 目錄存取權限控管</span><br />
<a href="http://huan-lin.blogspot.com/2010/01/subversion-folder-access-control.html">http://huan-lin.blogspot.com/2010/01/subversion-folder-access-control.html</a><br />
<br />
<br />Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-74894145378045458912013-12-19T18:41:00.005-08:002013-12-19T18:41:51.120-08:00為甚麼Java提供InterfaceSource: <a href="http://programming.im.ncnu.edu.tw/J_Chapter7.htm">http://programming.im.ncnu.edu.tw/J_Chapter7.htm</a><br />
<br />
<h3>
為甚麼Java提供Interface</h3>
雖然程式語言提供了基本資料型別,但由於各種應用都有其特定的資料結構需求,因此程式語言都提供使用者自訂型別的能力。型別自訂後,其使用的方法和基本資料型態類似。Class就是一種使用者自定的型別。Java提供了instanceof的保留字,用以判斷某reference所指到的物件,其型態和某Class是否相容:<br />
<pre>Object ref;
ref = <span style="color: #804040;"><b>new</b></span> Bird();
<span style="color: #804040;"><b>if</b></span> (ref <span style="color: #804040;"><b>instanceof</b></span> Animal) { <span style="color: blue;">// correct</span>
System.out.println(<span style="color: magenta;">"ref is currently pointing to an Animal Object."</span>);
}
</pre>
在物件導向的觀念裡,物件可以具有多個型別。例如「附有橡皮擦的鉛筆」,可以當成是「書寫工具」,也可以當成是「擦拭工具」。物件可有多種型別的觀念,不僅在日常生活中常見,在軟體開發上也有實際的需求。要使物件具有多種型別,可透過繼承來達成。例如Bird物件就同時具有Bird和Animal兩種型別。由於Java的Class只能有單一繼承,因此像「附有橡皮擦的鉛筆」同時具有「書寫工具」和「擦拭工具」兩種互不相關的型別,就無法透過Class的單一繼承來達成了。<br />
許多語言提供Class的多重繼承,但Java考量諸如下面的多重繼承問題,選擇不引進Class多重繼承:<br />
假設B繼承A,C繼承A,D又多重繼承B,C, 該語言又使用virtual function則<br />
<ul>
<li>如果B, C都有overwrite A的methodM方法, 而A ref指到D類別的物件, 請問透過ref傳遞methodM訊息時, 應該使用B還是C的methodM?</li>
</ul>
在不引進Class多重繼承的前提下,為了讓物件具有多種型態,Java提供了Interface(界面)的觀念。Interface可視為沒有實作的自訂型別,和Class用來作為Object的模板,有所不同。Class可以宣告實作多個Interface,而Interface之間可以有多重繼承。<br />
<h3>
Java有關Interface的語法</h3>
<ul>
<li>宣告Interface<pre><span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>interface</b></span> Listener {
<span style="color: seagreen;"><b>double</b></span> PI = <span style="color: magenta;">3.14149</span>; // 同public static final
<span style="color: seagreen;"><b>void</b></span> listen(); // 同public abstract
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>interface</b></span> Runnalbe {
<span style="color: seagreen;"><b>int</b></span> PERIOD = <span style="color: magenta;">10</span>;
<span style="color: seagreen;"><b>void</b></span> run();
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>interface</b></span> AnotherRun {
<span style="color: seagreen;"><b>int</b></span> PERIOD = <span style="color: magenta;">20</span>;
<span style="color: seagreen;"><b>void</b></span> run();
<span style="color: seagreen;"><b>int</b></span> run(<span style="color: seagreen;"><b>int</b></span>);
}
</pre>
注意上述函數宣告沒有{},也就是說沒有實作的意思。</li>
<li>Interface的繼承<pre><span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>interface</b></span> ActionListener <span style="color: seagreen;"><b>extends</b></span> Listener {
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>interface</b></span> MultiInterface <span style="color: seagreen;"><b>extends</b></span> Listener, Runnalbe {
}
</pre>
</li>
<li>Class實作Interface的宣告<pre><span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> A <span style="color: seagreen;"><b>implements</b></span> Listener {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> listen() {
}
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> B <span style="color: seagreen;"><b>implements</b></span> Listener, Runnable {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> listen() {
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> run() {
}
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> C <span style="color: seagreen;"><b>implements</b></span> MultiInterface {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> listen() {
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> run() {
}
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> D <span style="color: seagreen;"><b>extends</b></span> A <span style="color: seagreen;"><b>implements</b></span> Runnable, AnotherRun {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> run() {
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>int</b></span> run(<span style="color: seagreen;"><b>int</b></span> period) {
}
}
</pre>
</li>
</ul>
Interface如同Class一樣可以作為一種型態的宣告,因此如下的判斷都是正確的<br />
<pre>D ref = <span style="color: #804040;"><b>new</b></span> D();
ref <span style="color: #804040;"><b>instanceof</b></span> D; <span style="color: blue;">// true</span>
ref <span style="color: #804040;"><b>instanceof</b></span> Runnable; <span style="color: blue;">// true</span>
ref <span style="color: #804040;"><b>instanceof</b></span> AnotherRun; <span style="color: blue;">// true</span>
ref <span style="color: #804040;"><b>instanceof</b></span> A; <span style="color: blue;">// true</span>
ref <span style="color: #804040;"><b>instanceof</b></span> Listener; <span style="color: blue;">// true</span>
</pre>
Interface中宣告的變數具有以下特質<br />
<ul>
<li>public。所謂Interface(界面)指的是外界觀看某物件時,所能看到的表象以及溝通的管道,因此Interface內的成員一定是public。也就是說即便宣告時沒寫public關鍵字,Compiler也會幫我們加上去。</li>
<li>static。既然Interface沒有實作,就不可能透過Interface產生物件。換言之,Interface內的變數一定是屬於Class,而不屬於Object。</li>
<li>final。Interface可視為一種約定或契約,我們自然不希望裡面的variable可以隨便更改。</li>
</ul>
Interface中宣告的method具有以下特質<br />
<ul>
<li>public。同變數說明。</li>
<li>abstract。Interface沒有實作,裡面定義的method只是宣告而已。沒有實作的method,在Java裡用abstract這個關鍵字來表達。有關abstract的詳細說明,請見下一節</li>
</ul>
當Interface繼承多個Interface,或Class實作多個Interface時,如果有多個同名的函數或變數時,應該如何處理? 例如Runnable和AnotherRun這兩個界面都定義了變數PERIOD和方法run。<br />
<ul>
<li>相同變數名稱:由於interface內的變數具有static的性質,因此使用這些變數時,必須加上Interface的名稱才行,如Runnable.PERIOD,AnotherRun.PERIOD,因此不會造成任何混淆。</li>
<li>相同函數名稱:如果signature(參數個數,型態以及傳回值型態)完全相同,則Class只要實作一次即可,例如Runnable和AnotherRun均定義void run(),因此Class D只要實作一次就好了。如果同名函數符合Overloading,把它們分別當成不同的method即可。如果參數完全相同,但傳回值不同,則違反了Overloading的原則,會產生Compile Error。</li>
</ul>
<h3>
Abstract Class and Method</h3>
只有參數宣告,沒有實作的方法,稱為abstract method。某些情況下,雖然有實作,但我們希望強迫子類別必須override該方法時,也可以宣告為abstract method。Interface裡的方法一定沒有實作,因此必然為abstract method。<br />
如果Class裡有一個以上的abstract method,則該class必須宣告為abstract。有時候即使沒有abstract method,也可以宣告該class為abstract。我們不可以直接new該class的物件,只能new其子類別物件。<br />
<pre><span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>abstract</b></span> <span style="color: seagreen;"><b>class</b></span> AbstractExample {
<span style="color: seagreen;"><b>int</b></span> x;
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> <span style="color: seagreen;"><b>abstract</b></span> abstractMethod() {
}
<span style="color: seagreen;"><b>public</b></span> AbstractExample() {
x = <span style="color: magenta;">5</span>;
}
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> SubClass <span style="color: seagreen;"><b>extends</b></span> AbstractExample {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>void</b></span> abstractMethod() { <span style="color: blue;">// must override this method, or SubClass be declared as abstract class</span>
x = <span style="color: magenta;">10</span>;
}
}
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>class</b></span> Main {
<span style="color: seagreen;"><b>public</b></span> <span style="color: seagreen;"><b>static</b></span> <span style="color: seagreen;"><b>void</b></span> main(String[] argv) {
AbstractExample a = <span style="color: #804040;"><b>new</b></span> SubClass(); <span style="color: blue;">// correct</span>
a.abstractMethod(); <span style="color: blue;">// virtual function, call SubClass's abstractMethod</span>
a = <span style="color: #804040;"><b>new</b></span> AbstractExample(); <span style="color: blue;">// Compile error, you can't new abstract class</span>
}
}
</pre>
綜合以上所述,可列出以下幾點特徵<br />
<ul>
<li>具有abstract method的class必須宣告為abstract class。</li>
<li>繼承abstract class的子類別必須override所有父類別的abstract method,否則子類別也必須宣告為abstract class。</li>
<li>實作Interface A的Class必須實作A裡的所有method,否則必須宣告自己為abstract class。</li>
<li>不能直接new abstract class,只能new其非abstract class的子類別。</li>
</ul>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-88473074949597877822013-10-30T18:58:00.003-07:002013-10-30T18:59:23.071-07:00如何自动cradle wince模拟器<a href="http://m.blog.csdn.net/blog/chief1985/5429861#">http://m.blog.csdn.net/blog/chief1985/5429861#</a><br />
<br />
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
wince的模拟器有个Device Emulator Manager,这里面可以对模拟器进行cradle,之后Device Emulator Manager会是activesync和模拟器建立连接(DMA方式),之后wince的一些工具就可以用了。例如cecopy.exe,cedir.exe等,下面是一个列表:</div>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
cecopy.exe:拷贝文件到wince设备或模拟器(反过来也可以)<br />
cedel.exe:删除wince设备或模拟器上的文件<br />
cedir.exe:列举文件<br />
ceexec.exe:执行程序<br />
cegetinfo.exe:获取剩余空间信息<br />
cemkdir.exe:创建目录<br />
cereg.exe:注册表操作</div>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
这些工具的意义看名字就知道了,可以在<a href="http://www.microsoft.com/downloads/details.aspx?familyid=74473FD6-1DCC-47AA-AB28-6A2B006EDFE9&displaylang=en" style="color: black; text-decoration: none;">http://www.microsoft.com/downloads/details.aspx?familyid=74473FD6-1DCC-47AA-AB28-6A2B006EDFE9&displaylang=en</a>下载,这里面还有一些别的工具。</div>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
cradle一般的做法是Device Emulator Manager右键来选择。对于喜欢命令行一键搞定的程序员来说,cradle是一个比较麻烦的事,有没有可以在命令行里面cradle的方法呢?</div>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
答案是肯定的。在msdn里面,微软提供了一个对Device Emulator Manager编程的方法(<a href="http://msdn.microsoft.com/en-us/library/bb887543.aspx" style="color: black; text-decoration: none;">http://msdn.microsoft.com/en-us/library/bb887543.aspx</a>)。大家可以从<a href="http://download.microsoft.com/download/3/6/1/361517d0-2054-4d41-af95-36d18e141df5/DEMAutomationSetup.msi" style="color: black; text-decoration: none;" title="http://download.microsoft.com/download/3/6/1/361517d0-2054-4d41-af95-36d18e141df5/DEMAutomationSetup.msi">http://download.microsoft.com/download/3/6/1/361517d0-2054-4d41-af95-36d18e141df5/DEMAutomationSetup.msi</a>下载这个例子,然后进行编译,编译一般会报错,说找不到DEMComInterface.tlb,这就需要更改一些工程的Include目录。例子的作者将Device Emulator装在d盘,我们只需要将这个目录改为自己的Device Emulator安装地方就可以了。<strong>DEMAutomation</strong>支持如下命令,其中就有cradle 。Device Emulator Manager提供的API可以参考<a href="http://msdn.microsoft.com/en-us/library/bb531169.aspx" style="color: black; text-decoration: none;">http://msdn.microsoft.com/en-us/library/bb531169.aspx</a>。</div>
<table border="0" cellpadding="2" cellspacing="0" style="background-color: #ededed; color: black; font-family: Arial; font-size: 16px; line-height: 24px; margin: 0px; padding: 0px; width: 543px;"><tbody>
<tr style="margin: 0px; padding: 0px;"><td style="margin: 0px; padding: 0px;" valign="top" width="541"><div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
List :列举模拟器</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
bringtofront [VMID|Name] :窗口置顶</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
connect [VMID|Name] :连接</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
cradle [VMID|Name]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
clearsavestate [VMID|Name]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
getconfiguration [VMID|Name] [existing device configuration file]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
reset [VMID|Name]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
uncradle [VMID|Name]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
setconfiguration [VMID|Name] [device configuration file to create]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
shutdown [VMID|Name]</div>
<div style="margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<br /></div>
</td></tr>
</tbody></table>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
参考网址:</div>
<div style="background-color: #ededed; font-family: Arial; font-size: 16px; line-height: 24px; margin-bottom: 10px; margin-top: 10px; padding: 0px;">
<a href="http://msdn.microsoft.com/en-us/library/bb887543.aspx" style="color: black; text-decoration: none;">http://msdn.microsoft.com/en-us/library/bb887543.aspx</a><br />
<a href="http://msdn.microsoft.com/en-us/library/bb531169.aspx" style="color: black; text-decoration: none;">http://msdn.microsoft.com/en-us/library/bb531169.aspx</a><br />
http://social.msdn.microsoft.com/Forums/en-US/microsoftdeviceemu/thread/84fb02dc-2510-46b4-98e9-541297d555c4</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-65074625549678206302013-10-28T01:32:00.004-07:002013-10-28T01:33:54.962-07:00Windows Mobile App Development<h2>
Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> App Development </h2>
<div>
</div>
<div>
<ul style="background-color: white; border: 0px; color: #111111; font-family: 'Segoe UI', Arial, sans-serif; font-size: 14.399999618530273px; margin: 10px 0px; padding: 0px 0px 0px 40px;">
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43357/Windows-Mobile-App-Development-Part-1-Creating-you">Windows Mobile App Development Part 1: Creating your first application</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43376/Windows-Mobile-App-Development-Part-2-Device-Emula"><span style="color: purple;">Windows </span><span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; color: purple; font-weight: bold; margin: 0px; padding: 0px; text-decoration: none;">Mobile</span><span style="color: purple;"> App Development Part 2: Device Emulator and Device Emulator Manager</span></a><a href="http://www.codeproject.com/Articles/43376/Windows-Mobile-App-Development-Part-2-Device-Emula">Use Device Emulator and Cellular Emulator to test your application</a>s.</li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43378/Windows-Mobile-App-Development-Part-3-Basic-WinFor">Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> App Development Part 3: Basic WinForms App Development for Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span>Devices</a>Learn the basics of developing Windows Forms based applications for Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> devices.</li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43381/Windows-Mobile-App-Development-Part-4-Adding-Custo"><span style="color: purple;">Windows </span><span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; color: purple; font-weight: bold; margin: 0px; padding: 0px; text-decoration: none;">Mobile</span><span style="color: purple;"> App Development Part 4: Adding Custom Controls and Making Use of GPS Hardware</span></a><a href="http://www.codeproject.com/Articles/43381/Windows-Mobile-App-Development-Part-4-Adding-Custo">Learn to add custom controls and make use of GPS hardware in your applications.</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43383/Windows-Mobile-App-Development-Part-5-Intro-to-usi">Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> App Development Part 5: Intro to using SQL Server CE</a>Learn to access data through <code style="border: 0px; color: #990000; font-family: Consolas, 'Courier New', Courier, mono; font-size: 11pt; margin: 0px; padding: 0px;">DataSet</code>s and <code style="border: 0px; color: #990000; font-family: Consolas, 'Courier New', Courier, mono; font-size: 11pt; margin: 0px; padding: 0px;">SQLCeResultSet</code>s in your <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> apps.</li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43389/Windows-Mobile-App-Development-Part-6-Device-Secur">Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> App Development Part 6: Device Security and Application Deployment</a>Gain an understanding of device security, testing, and installation of your application on a Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span>device.</li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.codeproject.com/Articles/43396/Windows-Mobile-App-Development-Part-7-Mobile-Web-D">Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> App Development Part 7: <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> Web Development</a>Learn to create web based apps for <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> devices with AJAX support enabled using browser controls.</li>
</ul>
<h2 style="background-color: white; color: #ff9900; font-family: 'Segoe UI', Arial, sans-serif; font-size: 30px; font-weight: 200; margin: 20px 0px 11px; padding: 0px 0px 10px;">
Additional resources and references</h2>
<ul style="background-color: white; border: 0px; color: #111111; font-family: 'Segoe UI', Arial, sans-serif; font-size: 14.399999618530273px; margin: 10px 0px; padding: 0px 0px 0px 40px;">
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/windowsmobile/default.aspx">Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> Developer Center</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/library/bb158532.aspx" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">Introduction to development with Microsoft Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> 6</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/library/bb158509.aspx" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">Development Tools and Resources for Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> 6</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/library/bb158662.aspx" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">Code Samples for Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span></a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/windowsmobile/bb975138.aspx" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">How do I Build a .NET Compact Framework 3.5 Application for Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> 6 Devices</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://www.microsoft.com/events/EventDetails.aspx?CMTYSvcSource=MSCOMMedia&Params=~CMTYDataSvcParams%5E~arg%20Name=%22ID%22%20Value=%221032386646%22/%5E~arg%20Name=%22ProviderID%22%20Value=%22A6B43178-497C-4225-BA42-DF595171F04C%22/%5E~arg%20Name=%22lang%252" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">Introduction to Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> Device Development</a></li>
<li style="border: 0px; margin: 0px; padding: 0px;"><a href="http://msdn.microsoft.com/en-us/library/bb278115.aspx" style="border: 0px; color: purple; margin: 0px; padding: 0px; text-decoration: none;">What’s new for Developers in Windows <span class="search-highlight" style="border-bottom-color: rgb(255, 153, 0); border-bottom-style: dotted; border-width: 0px 0px 1px; font-weight: bold; margin: 0px; padding: 0px;">Mobile</span> 6</a></li>
</ul>
</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-4664330595155150322013-10-28T00:50:00.000-07:002013-10-28T00:50:33.621-07:00WMTF - UIAL Overview<div id="nsbanner">
<div id="TitleRow">
<h2>
WMTF - UIAL Overview</h2>
<h3>
UIAL Overview </h3>
</div>
</div>
The UI Abstraction Layer (UIAL) is a collection of C# classes that model
globalized access to an applications user interface. The Device Automation
Toolkit (DATK) provides C# classes for generic controls, whereas the <span style="background-color: yellow;">UIAL
exposes every individual instance of these controls for easy programmatic
access.</span> The UIAL hides details about each control, such as control ID or parent
window, from the tests that need to use these controls. In this way test cases
are isolated from changes in the UI - changes to details in UI controls only
require changes in the corresponding UIAL, not in the hundreds of test cases
that may require access to that control.<br />
<br />
<h3>
UIAL Components</h3>
Every application has its own set of UIAL projects - one for each supported
device. Within each UIAL project you will have 4 important components:<br />
<h4>
Dialog Classes</h4>
Every dialog in an application is exposed in a Dialog class.<br />
<ul>
<li>Derives from MobilityToolKit.MobilityDialog
</li>
<li>Contains details about the dialog itself such as control ID and parent
window.
</li>
<li>Exposes a DATK object for every control contained within the dialog. The
DATK objects are exposed as properties of the dialog class.
</li>
<li>Contains a VerifyAllControls() method that checks to see that each
non-optional control in the dialog can be found.
</li>
<li>Exposes a static instance of the IdnHolder class for the dialog to provide
access to localized strings found in the dialog.
</li>
<li>Recently updated versions expose a DATK ControlBinding object for every
control which gives you access to the WindowFinder used to bind to a control.
</li>
<li>Example:
%_WMTFROOT%\CalViewSample\AbstractionLayer\SmartPhone\CalViewMain.cs </li>
</ul>
<h4>
Application Class</h4>
Each application has one application class to hold all the dialog classes.
This is the starting point for locating any single UI element.<br />
<ul>
<li>Contains a Launch() method used to start the application using the UI.
</li>
<li>Contains a static property for each of the Dialog classes within the app.
</li>
<li>Example:
%_WMTFROOT%\CalViewSample\AbstractionLayer\SmartPhone\Microsoft.MobileDevices.AbstractionLayer.CalView.SmartPhone.cs
</li>
</ul>
<h4>
IdnHolder Classes</h4>
Each dialog in an application has an associated IdnHolder class to hold
localized resources for the dialog.<br />
<ul>
<li>Contains static IdnString instances for various string resources found in
the dialog.
</li>
<li>Example: CalViewSample\AbstractionLayer\SmartPhone\IdnHolders.cs </li>
</ul>
<h4>
UIAL self-test</h4>
Each UIAL contains a self-test suite that walks through the UI to validate
itself.<br />
<br />
<ul>
<li>Contains Begin() and End() methods for test setup and cleanup.
</li>
<li>Contains AbstractionLayerBVT() method which walks through the UI to call
VerifyAllControls() on every dialog in the application.
</li>
<li>Verification is very simple - the test passes if AbstractionLayerBVT()
completes without throwing an exception.
</li>
<li>Example:
%_WMTFROOT%\CalViewSample\AbstractionLayer\SmartPhone\Microsoft.MobileDevices.AbstractionLayer.CalView.SmartPhone.cs </li>
</ul>
<div>
<br /></div>
<div>
<div id="nsbanner">
<div id="TitleRow">
<h3>
How a UIAL is used </h3>
</div>
</div>
The UIAL for an application is used primarily by the Area Library for the
application. Briefly, the Area Library is a collection methods that accomplish
common UI tasks. For instance, the Inbox Area Library might contain a method
that configures a messaging account, while the Contacts Area Library might
contain a method to create a new contact. In order to accomplish these tasks the
Area Library has to interact with a large number of controls on the device.
Rather than specify the details of each control itself the Area Library simply
gets the DATK object for UI element that it is interested in from the UIAL and
then interacts with that object. For example, in order to select an item in the
sample CalView application, you would
use:<br />
<code>CalViewUIAL.CalViewMainDialog.MainListView.SetItemSelected(index,
true);</code>
The Area Library uses the UIAL to locate specific controls by narrowing down
the location from the app, to the dialog, to the control itself. The Area
Library only cares where the control is, not what properties it has, which
provides a level of isolation from changes in the UI.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Creating a UIAL </h3>
</div>
</div>
To a large extent the UIAL consists of generated code, but <u>area owner
expertise is needed to finish the process</u>. The following are the general steps
that are used to generate a UIAL. The work items below are needed to complete
the UIAL. Remember, when working with the UIAL, that most of the classes
representing dialogs will be automatically regenerated, so making changes to
these files is wasted work. For this reason when changes are needed in these
dialog classes you need to either add tags to the associated RCML files, or you
need to extend the class that needs changes and make the changes in the
inherited class. This is not the case with the classes representing
applications.<br />
<h4>
Generate RCML files for dialogs and windows.</h4>
Use <b><i>RcmlSnap </i></b>to take a snapshot for every window in your application. This
could take a few minutes per dialog. <br />
<h4>
Rename Controls in RCML to be Semantically Correct.</h4>
See the section on Editing the RCML to find out how to do this. This could
take about 15 - 30 minutes per file.<br />
<h4>
Generate your UIAL project.</h4>
<ul>
<li>Run <b><i>RCtoCS </i></b>to generate the cs files for your application windows, and
include them in your project.
</li>
<li>Run <b><i>UIALAppGen</i></b> to generate the code for the application and main dialog
classes. </li>
</ul>
<h4>
Expose IdnStrings.</h4>
<ul>
<li>IdnStrings are a way to globalize strings found in the product, so that test
automation can be easily globalized.
</li>
<li>Use <b><i>CeStrip </i></b>to extract the resource strings and IDs from each module in your
application.
</li>
<li><b><i>RCTOCS </i></b>will produce a skeleton file to hold your IDN strings called
<b>IdnHolders.cs</b> along with the other classes for the UIAL. In this file you will
find a class for each of the dialogs in your UIAL. Assuming that globalization
of your test automation is important, the task here is to populate these classes
with IdnString objects for strings that are exposed from your dialogs.
</li>
<li>Each class contains an example that will help you see how to create the
IdnString object. Depending on the size of your area and the number of strings
this could take a couple hours. </li>
</ul>
<h4>
UIAL Self-Test.</h4>
<br />
Every UIAL project has an auto generated self-test to smoke test the UIAL. A
self-test is a convenient way of ensuring a minimal level of correct
functionality of the UIAL. To implement this, you need to modify the test to
open every dialog before calling the validation method on that dialog. This will
take a couple hours at least.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Maintaining a UIAL </h3>
</div>
</div>
When UI or resource changes are made in the product the UIAL must be updated
to reflect these changes.<br />
<h4>
Re-run RcmlSnap to update the RCML files</h4>
<ul>
<li>Run <b><i>RcmlSnap </i></b>on a new or updated window. With updated windows, make sure
that <b><i>RcmlSnap </i></b>has access to the existing .rcml file, so that it can round-trip
changes you've made to it previously.</li>
</ul>
<h4>
Re-run RCTOCS to update the UIAL files</h4>
<ul>
<li>See <b><i>RCtoCS </i></b>for information on running the tool.</li>
</ul>
<h4>
Re-run CeStrip to get an updated list of resource strings. </h4>
<ul>
<li>See <b><i>CeStrip </i></b>for information on running the tool.
</li>
<li>Look at the IdnHolders.cs file for your UIAL project, a new class will be
added for every new RCML file.
</li>
<li>Add any new resource strings to the corresponding dialog classes in
IdnHolders.cs. </li>
</ul>
<h4>
Make sure the UIAL self-test passes</h4>
<br />
<ul>
<li>Build and the UIAL using Visual Studio.
</li>
<li>Run the UIAL self-test by deploying the UIAL project with the Tux.Net
project as the startup project. </li>
</ul>
<div>
<div id="nsbanner">
<div id="TitleRow">
<h3>
Tools used to work with a UIAL </h3>
</div>
</div>
These are the tools used when working with a UIAL:<br />
<ul>
<li><strong>RCtoCS</strong>: Tool generates UIAL Dialog classes based on RCML
files that have been decorated with specific tags.
</li>
<li><strong>RcmlSnap</strong>: Tool generates RCML based on the top-level window
currently displayed on a device.
</li>
<li><strong>GenFinder</strong>: Tool determines Datk.WindowFinder object that
should be used to locate a specific control on a device.
</li>
<li><strong>CeStrip</strong>: Tool which extracts resource strings in name=value
pairs from a Windows CE binary.
</li>
<li><strong>UIALAppGen</strong>: Tool generates a UIAL Application class based
on a number of UIAL Dialog class files. </li>
</ul>
<div>
<div id="nsbanner">
<div id="TitleRow">
<h3>
How to decorate RCML files for use by RCTOCS </h3>
</div>
</div>
In order to auto-generate abstraction layer code for your application, you
must first mark up the .RCML files that correspond to each dialog in the
application. <br />
RCML is XML, and the tool that reads it and converts it to C# code is looking
for a specific set of attributes that need to be added to the existing tags.
What follows is a list of the tags you want to change, the attributes you want
to add to each tag, and an example of each attribute. Attributes are
case-sensitive.<br />
<ul>
<li><strong><PAGE></strong>
<ul>
<li><strong>APPNAME</strong>="appName": This is the name of your app to be
appended to your project namespace when the C# code is generated.
</li>
<li><strong>FRIENDLYNAME</strong>="FindReplace": This will be the name of
the parent class containing all the child controls. Don't add "Dialog" as part
of the text, since that will be appended to your FRIENDLYNAME by the codegen
tools.
</li>
<li><strong>NAMESPACE</strong>="Microsoft.MobileDevices.AbstractionLayer.pOffice":
This is the parent namespace onto which APPNAME will be appended. Do not put the
value inside the APPNAME attribute into your NAMESPACE value. If you have a
parent app namespace, like pOffice or pOutlook, this is the right place to add
it.
</li>
<li><strong>PROJECT</strong>="PocketPC": Choices are PocketPC and
Smartphone.
</li>
<li><strong>DIRECTORY</strong>="\abstractionlayer\<appName>\<sub-appName>\<project>":
This is a relative path where the generated files will be copied.
</li>
<li><strong>CSHELP</strong>="The FindReplace dialog": This is a place to
put text that will be added to comments in the generated code.
</li>
<li><strong>PROCESSNAME</strong>="yourapp.exe" - (OPTIONAL): This
attribute will add a property to the dialog window finder that tells it to match
this dialog only if it is owned by the provided process name. This can be
helpful if the wrong dialog is being found when you try to initialize a specific
UIAL object.
</li>
<li><strong>SCOPE</strong>="private" - (OPTIONAL): This attribute will
cause a base and a derived UIAL class to be generated by rctocs. The derived
class can safely be hand-edited. Include this attribute if you need to override
any of the UIAL defaults. If you don't need to do this, leave this attribute
off.
</li>
<li><strong>BASECLASS</strong>="ClassName:(parameterString)" - (OPTIONAL):
This attribute specifies that the UIAL class generated for this dialog should be
derived from base class "ClassName" and that the constructor should pass
"paramaterString" to the base constructor. The base class needs to derive from
MobilityToolKit.MobilityDialog.
</li>
<li><strong>OBSCURED</strong>="true" - (OPTIONAL): This attribute will
force the .Visible property of the mainFinder object to be set to false. This
worked around a bug in the DATK that caused exceptions when a UIAL object is
initialized with the .Visible property set to true. This should no longer be
necessary.
</li>
<li><strong>IGNORETEXT</strong>="true" - (OPTIONAL): This attribute will
prevent rctocs from initializing the .Text property of the mainFinder object.
This is useful under circumstances where the main dialog's titlebar text changes
at runtime, or the auto-generated IDN value corresponding to the window title is
wrong, for whatever reason. NOTE: including this attribute will cause any value
inside the IDNTEXT attribute to not be generated.
</li>
<li><strong>IDNTEXT</strong>="YourDialogIdns.mainWindowTitle" -
(OPTIONAL): If present, the contents of this attribute will be used as the value
of the .Text property of the mainFinder object, in place of an auto-generated
IDN value. We recommend that the value you use here is a reference to a member
variable contained within the IdnHolder class that belongs to this dialog. An
instance of this class is already provided by rctocs, for your convenience, and
looks something like this:<br /><span style="font-family: Courier New; font-size: 11pt;"><span style="color: #999999;">///
<summary></span><span style="color: green;"><br /></span><span style="color: #999999;">///</span><span style="color: green;"> <span class="SpellE">IdnHolder</span>
object holding all the owner-maintained strings for <span class="SpellE">AlignDialog</span>.</span><span style="color: #999999;">///
</summary></span></span><span style="color: #000099; font-family: Arial; font-size: 11pt;"><br /></span><span style="font-family: Courier New;"><span style="font-size: 11pt;"><span style="color: blue;">public
<span class="SpellE">readonly</span></span> <span class="SpellE">AlignDialogIdnHolder</span> <span class="SpellE">AlignDialogIdns</span> = <span style="color: blue;">new</span> <span class="SpellE">AlignDialogIdnHolder</span>();</span></span> </li>
</ul>
</li>
<li><strong><BUTTON></strong>, <strong><CHECKBOX></strong>,
<strong><COMBOBOX></strong>, <strong><LABEL></strong>,
<strong><LISTBOX></strong>, <strong><SPINNER></strong>,
<strong><EDIT></strong> (the following attributes apply to all of these
tags)
<ul>
<li><strong>FRIENDLYNAME</strong>="ShowFormattingCheckBox": This will be
the name of the control in the generated code. You should append the control
type to 1 or 2 words describing the control.
</li>
<li><strong>CSHELP</strong>="Check this box to toggle formatting
display": This is a place to put text that will be added to comments in the
generated code.
</li>
<li><strong>OBSCURED</strong>="true" - (OPTIONAL): This attribute will
force the .Visible property of the control's WindowFinder object to be set to
false. This worked around a bug in the DATK that caused exceptions when a UIAL
object is initialized with the .Visible property set to true. This should no
longer be necessary.
</li>
<li><strong>OPTIONAL</strong>="true" - (OPTIONAL): This attribute will
prevent the control from being added to the dialog's VerifyAllControls() method.
This is useful for controls that are present and visible only under specific
circumstances at run-time.
</li>
<li><strong>MODIFIER</strong>="override" - (OPTIONAL): Use this to specify
a modifier for the control (override, virtual, public, private, etc.) where
public is the default if not specified.
</li>
<li><strong>IDNTEXT</strong>="YourDialogIdns.mainWindowTitle" -
(OPTIONAL): If present, the contents of this attribute will be used as the value
of the .Text property of the control's WindowFinder object. This is useful under
circumstances where identification by control ID alone is insufficient or
error-prone. We recommend that the value you use here is a reference to a member
variable contained within the IdnHolder class that belongs to this dialog. An
instance of this class is already provided by rctocs, for your convenience, and
looks something like this:<br /><span style="color: #999999; font-family: Courier New; font-size: 11pt;">/// <summary></span><span style="color: green; font-family: Courier New; font-size: 11pt;"><br /></span><span style="color: #999999; font-family: Courier New; font-size: 11pt;">///</span><span style="color: green; font-family: Courier New; font-size: 11pt;"> <span class="SpellE">IdnHolder</span> object
holding all the owner-maintained strings for <span class="SpellE">AlignDialog</span>.</span><span style="color: #999999; font-family: Courier New; font-size: 11pt;">/// </summary></span><span style="color: #000099;"><span style="font-family: Courier New; font-size: 11pt;"><br /></span></span><span style="font-family: Courier New; font-size: 11pt;"><span style="color: blue;">public <span class="SpellE">readonly</span></span> <span class="SpellE">AlignDialogIdnHolder</span> <span class="SpellE">AlignDialogIdns</span> = <span style="color: blue;">new</span> <span class="SpellE">AlignDialogIdnHolder</span>();</span> </li>
</ul>
</li>
</ul>
As you choose friendly names for your controls you want to postfix the DATK
Control type to the end of the variable name. For example, if you have a combo
box that sets the Type of a search you could name it SearchTypeComboBox. Or, if
you have a textbox that accepts a name you could name it NameTextBox. Here is
the list of DATK control names that should be used:<br />
<ul>
<li>Button
</li>
<li>CheckBox
</li>
<li>ColumnHeader
</li>
<li>ComboBox
</li>
<li>ContextMenu
</li>
<li>Control
</li>
<li>HtmlControl
</li>
<li>HtmlLink
</li>
<li>HtmlLinks
</li>
<li>Label
</li>
<li>LabelEditControl
</li>
<li>ListBox
</li>
<li>ListView
</li>
<li>ListViewItem
</li>
<li>MainMenu
</li>
<li>Menu
</li>
<li>MenuItem
</li>
<li>RadioButton
</li>
<li> RichEditBox
</li>
<li>SBTextBox
</li>
<li>SIPFloater
</li>
<li>SIPWindow
</li>
<li>SliderControl
</li>
<li>Softkeys
</li>
<li>TabControl
</li>
<li>TabPage
</li>
<li>TextBox
</li>
<li>ToolBar
</li>
<li>ToolBarButton
</li>
<li>VirtualListView </li>
</ul>
After your changes have been made, drag each file into Internet Explorer to
confirm that it parses correctly.<br />
</div>
</div>
</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-66662031295231595912013-10-27T19:35:00.001-07:002013-10-27T19:35:08.285-07:00Something relating to Windows CE/ Mobile<br />
<h2>
.NET Compact Framework</h2>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;">(Ref: MSDN - </span><a href="http://msdn.microsoft.com/zh-tw/library/f44bbwa1(v=vs.90).aspx">http://msdn.microsoft.com/zh-tw/library/f44bbwa1(v=vs.90).aspx</a>)</div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;">Microsoft .NET Compact Framework 是 Windows Mobile 和 Windows Embedded CE 裝置上不可或缺的元件,它可讓您建置和執行 Managed 應用程式以及使用 Web 服務。.NET Compact Framework 包含經最佳化的 Common Language Runtime (CLR) 和 .NET Framework 類別庫 (Class Library) 子集,後者支援如 Windows Communication Foundation (WCF) 和 Windows Form 等功能。此外,其中也包含專為 .NET Compact Framework 設計的類別。</span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;"><br /></span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;">.NET Compact Framework 可支援 Visual Basic 和 Visual C# 開發。目前它並不支援 C++ 開發。</span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;"><br /></span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;">.NET Compact Framework 繼承 Common Language Runtime 的完整 .NET Framework 架構,可用於執行 Managed 程式碼,還提供與 Windows CE 作業系統裝置的互通性 (Interoperability),使您可以存取原生 (Native) 的函式,並將您最喜歡的原生元件整合到應用程式中。</span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;"><br /></span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;">下列圖例摘要說明 .NET Compact Framework 平台架構。</span></div>
<div>
<span style="color: #2a2a2a; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 12.800000190734863px; line-height: 18px;"><br /></span></div>
<div>
<img alt=".NET Compact Framework 架構圖形" src="http://msdn.microsoft.com/dynimg/IC90954.gif" /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<h2>
.NETCF and Window Phone 7</h2>
Windows Phone 7 Series Application development platform is based on Microsoft Silverlight and XNA Framework – that are underneath powered by .NET Compact Framework (.NETCF). .NETCF provides the managed runtime environment for both Silverlight and XNA applications targeting Windows Phone 7.<br />
<br />
<br />
<br />
<h2>
Windows Mobile 5.0 - Emulators: copying files and ActiveSync</h2>
<div>
(<a href="http://blogs.msdn.com/b/johnkenn/archive/2005/05/19/420263.aspx">http://blogs.msdn.com/b/johnkenn/archive/2005/05/19/420263.aspx</a>)</div>
<div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;"><strong>Shared Folder</strong></span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;">The emulators allow you create a shared folder - a folder on your desktop PC, that appears as a Storage Card on the emulated device. Open up the Emulator's properties from "File / Configure.." and then select a Shared Folder from the textbox.</span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;"><strong>ActiveSync</strong></span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;">Now this is really cool. The new emulators can actually connect to the copy of ActiveSync running on your desktop, and they behave like a real, physical device. This means you can select Explore, and wander through their file system copying and pasting files. This is by far the simplest way to copy your support files to your emulator. You can even copy CAB files, and test your new installers (something I'll cover in another blog entry very soon).</span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;">To get ActiveSync working, first configure ActiveSync itself (you'll need the very latest version), by opening "Connection Settings", and changing the pull-down under "Allow connections to one of the following" to "DMA". DMA is "Direct Memory Access", and allows the emulator to create a direct connection with ActiveSync without any extra wires.</span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;">Next, from within Visual Studio, open the Tools menu and select "Device Emulator Manger". From within this dialog, find the current emulator you are using, right click, and select "Cradle".</span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<span style="font-family: Arial; font-size: x-small;">Then on the emulated device itself, open the ActiveSync tool, and select "Connect via DMA..." from the menu option.</span></div>
<div class="MsoNormal" style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px; margin-bottom: 0.0001pt !important;">
<br /></div>
<span style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px;"><span style="font-family: Arial; font-size: x-small;">And that's it! You should see ActiveSync starting as if the emulator was a real device. Now if you click Explore on the ActiveSync dialog, you can drill down to whatever location you need to copy and paste files.</span></span></div>
<div>
<span style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px;"><span style="font-family: Arial; font-size: x-small;"><br /></span></span></div>
<div>
<span style="background-color: #ced5db; color: #333333; font-family: 'Segoe UI', 'Lucida Grande', Verdana, Arial, Helvetica, sans-serif; font-size: 15.199999809265137px; line-height: 22.700000762939453px;"><span style="font-family: Arial; font-size: x-small;"><br /></span></span></div>
<h2>
Step by Step: Using Microsoft Device Emulator In-Depth in Your Application Development Experienc</h2>
<div>
(<a href="http://msdn.microsoft.com/en-us/library/bb278114.aspx">http://msdn.microsoft.com/en-us/library/bb278114.aspx</a>)</div>
<br />
<br />
<h2>
Getting Started with Building Windows Mobile Solutions with Visual Studio and Windows Mobile 6 SDK</h2>
<div>
<a href="http://msdn.microsoft.com/en-us/library/dd721907.aspx">http://msdn.microsoft.com/en-us/library/dd721907.aspx</a></div>
<div>
<br /></div>
<h2>
Windows 7 - Using the Windows Mobile Emulator</h2>
<div>
<a href="http://www.petenetlive.com/KB/Article/0000241.htm">http://www.petenetlive.com/KB/Article/0000241.htm</a><br />
<br />
<br /></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-62322125516319177262013-10-27T19:34:00.001-07:002013-11-20T17:21:50.661-08:00Windows Mobile Test Framework Overview<div id="nsbanner">
<div id="bannerrow1">
<table cellspacing="0" class="bannerparthead">
<tbody>
<tr id="hdr">
<td class="runninghead" nowrap=""><h2>
Windows Mobile Test Framework Overview</h2>
</td>
<td class="product" nowrap=""></td></tr>
</tbody></table>
</div>
<div id="TitleRow">
<h3>
Framework Layers</h3>
</div>
</div>
This document will give a high level overview of the pieces that make up the
Windows Mobile Test Framework. This framework is made primarily of .Net class
libraries which are used to automate interaction with the device UI.<br />
These class libraries are broken out into separate layers with differing
purposes. The layers are shown in the diagram below, which illustrates how the
different pieces are built on top of one another. Tux.Net, not shown here,
simply executes code from the test suites, but is built primarily on Logging and
Utils.Net components. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1ZBV6t2b5UUakCGaMb-1RX0Jx-mQ1S9CcE8xtqKBi0OIj6bcNXTubrwwNZeou5dvlkwX0F9xzsgjoeVZdxVAOFTPnVmROebT8WTejJb8TnlOT0NTToDZ-ZQVchnl0ZYsbZyh4If6Tqj5X/s1600/WMTF+layer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="368" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1ZBV6t2b5UUakCGaMb-1RX0Jx-mQ1S9CcE8xtqKBi0OIj6bcNXTubrwwNZeou5dvlkwX0F9xzsgjoeVZdxVAOFTPnVmROebT8WTejJb8TnlOT0NTToDZ-ZQVchnl0ZYsbZyh4If6Tqj5X/s640/WMTF+layer.png" width="640" /></a></div>
<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Logging </h3>
</div>
</div>
The Logging library contains classes associated with logging test data and
test result management. The Logging library is generic enough that it can be
used by any tool, not just .Net test automation, and it can even be used by
desktop tools. The main interfaces into the Logging library for .Net Automation
are GlobalLogger and GlobalLogResultManager. <br />
GlobalLogger is used for logging text and comments to log files during
testing. It is used by Mtk, UIAL, Area Library, and Test components. It frees
you from worrying about where and how to log data - it will make sure your data
gets in the right file, where the "right" file is defined by whatever file the
GlobalLogger has been configured to deliver to. GlobalLogger also has the notion
of logging levels so that you can assign varying importance to different
comments. For instance, UIAL comments may be logged at debug level and only
turned on during debugging, whereas comments at the test level would always be
logged. GlobalLogResultManager is used for keeping track of the result of your
test - Pass or Fail - as you perform multiple steps in each test case. It is
used almost exclusively in the Test and Area Library layers.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Utils.Net </h3>
</div>
</div>
Utils.Net is a toolbox of common device oriented code that can be reused in
any automation component including MTK, UIAL, Area Libraries, and Tests. The
.Net Framework provides a very rich library of code for desktop tools to reuse,
but the .Net Compact Framework contains only a subset of this library. Utils.Net
includes many things missing from the .Net Compact Framework that we need on a
regular basis to work with and test our devices. As we have need of these
missing library components we crate and add them to Utils.Net.<br />
<br />
Utils.Net holds code that isn't just useful to .Net Automation, but
potentially for anyone who is working with the .Net Compact Framework. In
addition, Utils.Net builds for the desktop to facilitate sharing code between
device and desktop.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Device Automation Toolkit </h3>
</div>
</div>
The Device Automation ToolKit (DATK) is primarily a C# library of classes
that mirror controls on our devices. Most classes in DATK map to a control found
in our devices, such as edit boxes, list views, and buttons. In addition to
these control classes there is the WindowFinder class, which is the key to
working with DATK. It allows you to specify the characteristics (such as type,
text, and owning process) of the control you are looking for on the device, and
bind that control to a DATK control object. Using the DATK control object you
can get information about the control on the device such as the text,
selected/unselected status, focused/unfocused status, and more. Even more
importantly you can use the DATK control object to interact with the control on
the device. In this way we can use DATK to automate the use of UI on the
device.<br />
<br />
The DATK does not currently depend on the Logging or Utils.Net
components.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Mobility Toolkit </h3>
</div>
</div>
The Mobility ToolKit (MTK) is simply an Windows Mobile specific extension to
the DATK class library. Because DATK is dedicated to all graphical Windows CE
platforms, it has been kept it free of classes representing controls that are
not provided as part of the Windows CE operating system. For instance, Softkeys
are a very important component in Mobile devices but do not exist in Windows CE
itself. We have placed Windows Mobile specific DATK controls into their own
library, the MTK. In addition, MTK holds some other classes which are useful to
automation. ApplicationManager is the most obvious example of this, which we use
to launch applications on Windows Mobile devices. <br />
<br />
The MTK has many classes that derive from DATK classes, and it also depends
heavily on Logging and Utils.Net.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Abstraction Layer </h3>
</div>
</div>
DATK provides classes for most of the UI elements found on our devices.
However, in order to initialize one of these objects to an actual UI element at
the Datk layer, you need to use the Datk WindowFinder class. A WndowFinder is
provided search criteria such as class name, title, label, control ID, etc and
then asked to find the control that matches the criteria it was given. The
search criteria are especially sensitive to design and implementation change of
the UI, and so this warrants a mode of abstraction to help protect test
automation from these changes. <br />
The UI Abstraction Layer, or UIAL, is meant to encapsulate all controls on
every form of an application so that the automation developer does not have to
determine the properties of every control he uses or use WindowFinder to get
that control. It is a C# library that provides a 1:1 mapping between particular
controls on the device and DATK control objects. There should be a UIAL for
every application, and there should be a property in each UIAL that maps to
every control found in that application. This centralizes the work of
characterizing each control so that when elements of the UI change only the UIAL
needs to change, not all of the code that automates that element.<br />
<b>An application developer must create a UIAL component in order to make
their application testable. </b>Anyone can do this for most applications,
actually, but only the application developer can reliably ensure that the UIAL
adheres to a contract between the automation and the application as the
application changes.<br />
<br />
The UIAL depends heavily on DATK, MTK, Utils.Net, and Logging. It is used
almost exclusively by the Area Libraries. The UIAL is largely tool-generated
code, not hand-written code.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Area Libraries </h3>
</div>
</div>
The UIAL is used to hide the complexity of finding controls in DATK, but it
does not combine the use of these controls into common usage scenarios. This is
what the Area Libraries are for - providing a library of routines that act out
common UI scenarios on the device.<br />
The goal of the libraries is to make test script writing as straightforward
as typing in descriptions of manual test case steps. The code is very generally comparable to the aggregate steps taken in manual testing. For instance, the
first step in a test case for Microsoft Word would be open Word.exe, which
corresponds to a series of UI based commands: click Start, click Programs File, Scroll Down, Click Word. "Open Word" might be implemented in the Area Library,
so that the test case doesn't need to deal with the specifics of what it takes
to open Microsoft Word. The Area Library also abstracts away the notions of a
Windows Mobile SKU (Professional, Classic, Standard) to provide functions that
act appropriately depending on which version is being used. <br />
<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Test Cases </h3>
</div>
</div>
In our .Net Automation Framework test suites are simply C# classes that
inherit from a class called TestSuite. Tests are simply C# methods in these
classes decorated with an attribute that designates them as test cases. Tests
are generally organized into test suites by feature. Tests rely almost solely on
the area library for functionality, and consist of not much more than a series
of calls into the area library, to string together common UI scenarios into a
single test scenario. Tests do not call into the UIAL unless the test is
specific to a particular kind of Windows Mobile device, which should be fairly
rare, depending on the application being tested.<br />
<b>A Test developer should create a test assembly to host test cases for
their application.</b><br />
<br />
The Tests depend heavily on Area Libraries, for scenario and object
creational patterns, and Logging. They occasionally (rarely in theory) need to
use UIAL, MTK, or DATK.<br />
<br />
<div id="nsbanner">
<div id="TitleRow">
<h3>
Tux.Net </h3>
</div>
</div>
Tux.Net is a C# program, specifically a test harness, used to run tests on
devices. When run from a device with command line or a configuration file it
will locate the specified test suites and tests inside managed assemblies and
execute them. It has many options for specifying what tests or suites will be
run, and how they should be run (for instance, should they be shuffled or
repeated). Tux.Net looks for classes inheriting the TestSuite class and methods
with the TestCase attribute to know what it can run, and it can run any assembly
meeting these requirements. <br />
<br />
Tux.Net executes Tests, but it is built primarily with the Logging library
and Utils.Net.<br />
<br />
Source: WMTF DocumentMorrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-3750402084301780822013-06-10T23:12:00.001-07:002013-06-10T23:12:10.378-07:00Android MonkeyReference:<br />
1.<a href="http://developer.android.com/tools/help/monkey.html">http://developer.android.com/tools/help/monkey.html</a><br />
2.<a href="http://kaijie-chang.blogspot.tw/2011/07/android-monkey.html">http://kaijie-chang.blogspot.tw/2011/07/android-monkey.html</a><br />
3. <a href="http://jjnnykimo.pixnet.net/blog/post/35187781-android-monkey-test-%E4%BD%BF%E7%94%A8%E8%AE%8A%E6%95%B8%E4%BB%8B%E7%B4%B9">http://jjnnykimo.pixnet.net/blog/post/35187781-android-monkey-test-%E4%BD%BF%E7%94%A8%E8%AE%8A%E6%95%B8%E4%BB%8B%E7%B4%B9</a><br />
4. <a href="http://android-test-tw.blogspot.tw/2012/10/android-monkey-automation-test.html">http://android-test-tw.blogspot.tw/2012/10/android-monkey-automation-test.html</a><br />
<br />Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-57151599177338149662013-06-04T23:26:00.000-07:002013-06-04T23:26:34.504-07:00Flask & SELinux<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Flask was developed to work through some of the inherent problems with a MAC architecture. Traditional MAC is closely integrated with the multi-level security (MLS) model. Access decisions in MLS are based on clearances for subjects and classifications for objects, with the objective of no read-up, no write-down . This provides a very static lattice that allows the system to decide by a subject's security clearance level which objects can be read and written to. The focus of the MLS architecture is entirely on maintaining confidentiality.<br /><br />The inflexible aspect of this kind of MAC is the focus on confidentiality. The MLS system does not care about integrity of data, least privilege, or separating processes and objects by their duty, and has no mechanisms for controlling these security needs. MLS is a mechanism for maintaining confidentiality of files on the system, by making sure that unauthorized users cannot read from or write to them.<br /><br />Flask solves the inflexibility of MLS-based MAC by separating the policy enforcement from the policy logic, which is also known as the security server. In traditional Flask, the security server holds the security policy logic, handling the interpretation of security contexts. Security contexts or labels are the set of security attributes associated with a process or an object. Such security labels have the format of <i><user>:<role>:<type></i>, for example, <i>system_u:object_r:httpd_exec_t</i>. The SELinux user system_u is a standard identity used for daemons. The role <i>object_r</i> is the role for system objects such as files and devices. The type <i>httpd_exec_t</i> is the type applied to the httpd executable /usr/sbin/httpd.<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUymBjQeo8OeeALr7MM1RsOUw2SfCyz2e4oHX26AaONG6Ul4CP0J0Thkd5J7UQB8UO-rU4oBFHQDjc3RFJ8RttcXv5dWx_hBx86Eui96RFlGyNP0JupgVz39sSyVF0HfXQFvu0MQdG3H3d/s1600/flask.png" imageanchor="1"><img border="0" height="602" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUymBjQeo8OeeALr7MM1RsOUw2SfCyz2e4oHX26AaONG6Ul4CP0J0Thkd5J7UQB8UO-rU4oBFHQDjc3RFJ8RttcXv5dWx_hBx86Eui96RFlGyNP0JupgVz39sSyVF0HfXQFvu0MQdG3H3d/s320/flask.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Above picture shows the Flask architecture. In this operation, standard DAC has occurred, which means the subject already has gained access to the object via regular Linux file permissions based on the UID<a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/SELinux_Guide/selg-chapter-0013.html#FTN.AEN592">[1]</a>. The operation can be anything: reading from or writing to a file/device, transitioning a process from one type to another type, opening a socket for an operation, delivering a signal call, and so forth<span style="background-color: white; color: #333333; font-family: helvetica, arial, sans-serif; font-size: 12pt; line-height: 25.454544067382813px;">.</span></div>
<ol>
<li>A subject, which is a process, attempts to perform an operation on an object, such as a file, device, process, or socket.</li>
<li>The policy enforcement server gathers the security context from the subject and object, and sends the pair of labels to the security server, which is responsible for policy decision making.</li>
<li>The policy server first checks the AVC, and returns a decision to the enforcement server.</li>
<li>If the AVC does not have a policy decision cached, it turns to the security server, which uses the binary policy that is loaded into the kernel during initialization. The AVC caches the decision, and returns the decision to the enforcement server, that is, the kernel.</li>
<li>If the policy permits the subject to perform the desired operation on the object, the operation is allowed to proceed.</li>
</ol>
If the policy does not permit the subject to perform the desired operation, the action is denied, and one or more avc: denied messages are logged to$AUDIT_LOG, which is typically /var/log/messages in Red Hat Enterprise Linux.With the security server handling the policy decision making, the enforcement server handles the rest of the tasks. In this role, you can think of the enforcement code as being an object manager. Object management includes labeling objects with a security context, managing object labels in memory, and managing client and server labeling.<br /><div>
<br /></div>
<div>
<br /></div>
<div>
Reference:</div>
<div>
<a href="https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/SELinux_Guide/selg-chapter-0013.html">https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/SELinux_Guide/selg-chapter-0013.html</a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-71572379360705530962013-06-02T20:42:00.003-07:002013-06-02T23:19:15.868-07:00Android SELinux: binder & socket<h3>
Introduction of binder</h3>
<span style="background-color: #f7f7f7; font-family: Verdana, sans-serif; font-size: 14px;"><br /></span>
Inter Process Communication (IPC) has been a part of Android since 1.0, and yet most of us take it for granted.<br />
<br />
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFdiaz33rkMd6zF_VcdG5Dhg947jz4lEWVU7cC2JvAurCADvikE7g3kvM63bEN-dH2gFG_3W5CVrgO1iKIf5TBAI0dJYeESHCCWkRPY8ush_UEAq-fmDE8lD68lWJWX9LUPPj7A48Dj6tF/s1600/binder.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFdiaz33rkMd6zF_VcdG5Dhg947jz4lEWVU7cC2JvAurCADvikE7g3kvM63bEN-dH2gFG_3W5CVrgO1iKIf5TBAI0dJYeESHCCWkRPY8ush_UEAq-fmDE8lD68lWJWX9LUPPj7A48Dj6tF/s1600/binder.png" height="434" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
While processes cannot directly invoke operations (or read/write data) on other processes, the kernel can, so they make use of the Binder driver. Binder driver is exposed via <i>/dev/binder</i> and offers a relatively simple API based on <i>open, release, poll,mmap, flush, and ioctl</i> operations.<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0qMIAHjDyUbUrGjEMZbe0inlrZXbpVlg7nDHlDu4g59OfKk3GWA8i0GZl7LlXprdZHD9REqvizuy-OL-uRlYfZ80yU9ck9123xfuxgpqx8TFNYeRqLS3URHp_sCP2c-oMqP9UkFHSzGaS/s1600/fibonacci.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0qMIAHjDyUbUrGjEMZbe0inlrZXbpVlg7nDHlDu4g59OfKk3GWA8i0GZl7LlXprdZHD9REqvizuy-OL-uRlYfZ80yU9ck9123xfuxgpqx8TFNYeRqLS3URHp_sCP2c-oMqP9UkFHSzGaS/s1600/fibonacci.png" height="406" width="640" /></a></div>
<div>
<br /></div>
<h3>
Security Mechanism of binder</h3>
<div>
Malicious applications can make use of binder mechanism to get unauthorized data, SEforAndroid implements security control to check the permissions. The details are as following:</div>
<div>
<br /></div>
<div>
1. Declare a class for kernel policy in <i>external/sepolicy/access_vectors</i></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">class binder</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">{</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> impersonate</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> call</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> set_context_mgr</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> transfer</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> receive</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">}</span></div>
<div>
<br /></div>
<br />
Impersonate:該 process 是否可以代表另一process使用binder. <span style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;"> </span>kernel (selinux/hooks.c) checks permission in selinux_binder_transaction call.<br />
Call:Perform a binder IPC to a given target process (can A call B?).<br />
Set_context_mgr:是否可以将自己註册成Context Manager. Can A set the context manager to B, where normally <i>A == B</i>.See policy module <i>servicemanager.te</i>.<br />
Transfer:是否可以傳遞某類型的binder引用到其他process. Transfer a binder reference to another process (can A transfer a binder reference to B?).<br />
Receive:是否可以接收某類型binder引用.<br />
<div>
<br /></div>
<div>
2. TE中使用 macro 進行配置 binder_use、binder_call、binder_transfer、binder_service</div>
<div>
Ex:</div>
<div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># binder_call(clientdomain, serverdomain)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># Allow clientdomain to perform binder IPC to serverdomain.</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">define(`binder_call', `</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># First we receive a Binder ref to the server, then we call it.</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">allow $1 $2:binder { receive call };</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># Receive and use open files from the server.</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">allow $1 $2:fd use;')</span></div>
</div>
<div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">例如</span><span style="font-family: 'Times New Roman', serif;">servicemanager</span><span style="font-family: 宋体;">:</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 'Times New Roman', serif;">allow servicemanager self:binder set_context_mgr;</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 'Times New Roman', serif;">allow servicemanager domain:binder { receive transfer };</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">配置表示了</span><span style="font-family: 'Times New Roman', serif;">servicemanager</span><span style="font-family: 宋体;">可以将自己设置为</span><span style="font-family: 'Times New Roman', serif;">context manager</span><span style="font-family: 宋体;">,并且它可以对所有</span><span style="font-family: 'Times New Roman', serif;">domain</span><span style="font-family: 宋体;">执行</span><span style="font-family: 'Times New Roman', serif;">receive</span><span style="font-family: 宋体;">和</span><span style="font-family: 'Times New Roman', serif;">transfer</span><span style="font-family: 宋体;">的操作。</span></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
3. 於 source code中增加相關的操作函數</div>
<div>
Ex.<i> kernel/goldfish/security/selinux/hooks.c</i> defines four functions</div>
<div>
<br /></div>
<div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">.binder_set_context_mgr =selinux_binder_set_context_mgr,</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">.binder_<a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK11" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK10" style="color: #006699; text-decoration: underline;"></a>transaction = selinux_binder_transaction,</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">.binder_transfer_binder =selinux_binder_transfer_binder,</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">.binder_transfer_file = selinux_binder_transfer_file,</span></div>
</div>
<div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">首先看一下第一个函数,其的实现原理就是去</span><span style="font-family: 'Times New Roman', serif;">AVC</span><span style="font-family: 宋体;">中查询当前的</span><span style="font-family: 'Times New Roman', serif;">sid</span><span style="font-family: 宋体;">是否设置了</span><span style="font-family: 'Times New Roman', serif;">context_mgr</span><span style="font-family: 宋体;">的权限,如果未经授权,则禁止此次操作。</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">static int selinux_binder_set_context_mgr(struct task_struct *mgr)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">{</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> u32 mysid = current_sid();</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> u32 mgrsid = task_sid(mgr);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> return avc_has_perm(mysid, mgrsid, SECCLASS_BINDER, BINDER__SET_CONTEXT_MGR, NULL);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">}</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<br /></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">对于</span><span style="font-family: 'Times New Roman', serif;">transaction</span><span style="font-family: 宋体;">的控制也是类似,在</span><span style="font-family: 'Times New Roman', serif;">binder_transaction</span><span style="font-family: 宋体;">中增加</span><span style="font-family: 'Times New Roman', serif;">hook</span><span style="font-family: 宋体;">,用来检查本次调用的权限,其中也是同样在</span><span style="font-family: 'Times New Roman', serif;">AVC</span><span style="font-family: 宋体;">中查询权限。</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">static void <a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK7" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK6" style="color: #006699; text-decoration: underline;"></a>binder_transaction(struct binder_proc *proc,</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> struct binder_thread *thread,</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> struct binder_transaction_data *tr, int reply)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 宋体; font-size: 9pt;">……</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> if (security_binder_transaction(proc->tsk, target_proc->tsk) < 0) {</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> return_error = BR_FAILED_REPLY;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> goto err_invalid_target_handle;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> }</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 宋体; font-size: 9pt;">……</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<br /></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">static int selinux_binder_transaction(struct task_struct *from, struct task_struct *to)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">{</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> u32 mysid = current_sid();</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> u32 fromsid = task_sid(from);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> u32 tosid = task_sid(to);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> int rc;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<br /></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> if (mysid != fromsid) {</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> rc = avc_has_perm(mysid, fromsid, SECCLASS_BINDER, BINDER__<a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK13" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK12" style="color: #006699; text-decoration: underline;"></a>IMPERSONATE, NULL);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> if (rc)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> return rc;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> }</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<br /></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"> return avc_has_perm(fromsid, tosid, SECCLASS_BINDER, BINDER__CALL, NULL);</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">}</span></div>
</div>
<div>
<br /></div>
<h3>
socket</h3>
<div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">在</span><span style="font-family: 'Times New Roman', serif;">SEAndroid</span><span style="font-family: 宋体;">中</span><span style="font-family: 'Times New Roman', serif;">service sockets</span><span style="font-family: 宋体;">的權限同樣受到管理。</span><span style="font-family: 'Times New Roman', serif;">Init process </span><span style="font-family: 宋体;">在創建</span><span style="font-family: 'Times New Roman', serif;">service</span><span style="font-family: 宋体;">附属</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">的同時,根據 </span><span style="font-family: 'Times New Roman', serif;">file_contexts </span><span style="font-family: 宋体;">查詢當前</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">的</span><span style="font-family: 宋体;">權限</span><span style="font-family: 宋体;">,並將信息加入到</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">的</span><span style="font-family: 'Times New Roman', serif;">security context</span><span style="font-family: 宋体;">中,啟動後的</span><span style="font-family: 宋体;">權限</span><span style="font-family: 宋体; font-size: 14.399999618530273px; line-height: 25.200000762939453px; text-indent: 21pt;">如下所示:</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">srw------- system system u:object_r:<a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK9" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK8" style="color: #006699; text-decoration: underline;"></a>installd_socket:s0 installd</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">srw-rw-rw- root root u:object_r:keystore_socket:s0 keystore</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">srw-rw---- root system u:object_r:netd_socket:s0 netd</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">srw-rw-rw- root root u:object_r:property_socket:s0 property_service</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">srw-rw---- root radio u:object_r:rild_socket:s0 rild</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 宋体; font-size: 9pt;">……</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">关于</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">使用权限的配置可以简单的使用两个 macro 定義 </span><span style="font-family: 'Times New Roman', serif;">unix_socket_connect</span><span style="font-family: 宋体;">、</span><span style="font-family: 'Times New Roman', serif;">unix_socket_send</span><span style="font-family: 宋体;">,他们分别对应着 </span><span style="font-family: 'Times New Roman', serif;">TCP </span><span style="font-family: 宋体;">和 </span><span style="font-family: 'Times New Roman', serif;">UDP </span><span style="font-family: 宋体;">类型的</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">访问。</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># <a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK15" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK14" style="color: #006699; text-decoration: underline;"></a>unix_socket_connect(clientdomain, socket, serverdomain)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># <a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK17" style="color: #006699; text-decoration: underline;"></a><a href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" name="OLE_LINK16" style="color: #006699; text-decoration: underline;"></a>Allow a local socket connection from clientdomain via</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;"># socket to serverdomain</span><span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">.</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">define(`unix_socket_connect', `</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">allow $1 $2_socket:sock_file write;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">allow $1 $3:unix_stream_socket connectto;</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">')</span></div>
<div class="MsoNormal" style="background-color: #efefef; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 25.200000762939453px; margin: 0in 0in 0pt; padding: 0px; text-indent: 21pt;">
<span style="font-family: 宋体;">对于规则的配置只需如下,他表示了</span><span style="font-family: 'Times New Roman', serif;">adbd domain</span><span style="font-family: 宋体;">的</span><span style="font-family: 'Times New Roman', serif;">subject</span><span style="font-family: 宋体;">可以通过</span><span style="font-family: 'Times New Roman', serif;">vold_socket</span><span style="font-family: 宋体;">类型的</span><span style="font-family: 'Times New Roman', serif;">socket</span><span style="font-family: 宋体;">访问</span><span style="font-family: 'Times New Roman', serif;">vold</span><span style="font-family: 宋体;">的</span><span style="font-family: 'Times New Roman', serif;">domain</span><span style="font-family: 宋体;">。</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<span style="color: #393939; font-family: 'Courier New'; font-size: 9pt;">unix_socket_connect(adbd, vold, vold)</span></div>
<div class="MsoNormal" style="background-color: #ccffcc; font-family: Helvetica, Tahoma, Arial, sans-serif; font-size: 14.399999618530273px; line-height: 12.25pt; margin: 0in 0in 0pt; padding: 0px;">
<br /></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
reference:</div>
<div>
<a href="http://www.iteye.com/topic/1129994">http://www.iteye.com/topic/1129994</a></div>
<div>
<br /></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-26384020295935459922013-05-26T19:46:00.001-07:002013-06-24T19:35:18.060-07:00SELinux MAC Policy Files<dl style="margin-bottom: 0.5em; margin-top: 0.2em;"><dd style="margin-bottom: 0.1em; margin-left: 2em;">
<dl style="margin-bottom: 0.5em; margin-top: 0.2em;"><br />The MAC policy configuration files are contained in the external/sepolicy directory, however there may also be policy configuration files to enable specific device features under <i>device/<vendor> </i>directories (see the <a href="http://selinuxproject.org/page/NB_SEforAndroid_1#Building_the_Policy">Building the Policy</a> section). Once generated, the policy and its supporting configuration files are installed on the device as part of the build process.<br /><br /><br />The following files are used to build the kernel binary policy file that will be named sepolicy and installed by default in the root directory. An updated policy may be installed at<i> /data/security/sepolicy</i>. The initialisation or policy reload process will always check for a policy at <i>/data/security/sepolicy </i>first and then if not present at <i>/sepolicy</i>. The policy files consist of the following:<br /><br /><span style="font-family: Courier New, Courier, monospace;">access_vectors, security_classes </span><br /><br />These have been modified to support the new SE for Android classes and permissions.</dl>
<dl style="margin-bottom: 0.5em; margin-top: 0.2em;">Access decisions specify whether or not a permission is granted for a given pair of SIDs and class. Each object class has a set of associated permissions defined to control operations on objects with that class. These permission sets are represented by a <span style="background-color: yellow;">bitmap </span>called an <span style="background-color: yellow;">access vector</span>. The corresponding constants for permissions are defined in the automatically generated header file <i>av_permissions.h (external/libsepol/src </i>and<i> kernel/goldfish/security/selinux)</i>. This file looks like below figure:</dl>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgU6WiIDAeVBpNhfQgQAESS_VdBUoUbpw5t_VrNxsgNNk_UrQ5MRv9xpT9j9W1M4SV_3ymSBmDDA09qM2TCXTKAJIemuo8f3Epix8SWeF2g2ToITD7xZP4L2scyCT1RoQe_rBo1VAGDo1Jp/s1600/av-permission.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgU6WiIDAeVBpNhfQgQAESS_VdBUoUbpw5t_VrNxsgNNk_UrQ5MRv9xpT9j9W1M4SV_3ymSBmDDA09qM2TCXTKAJIemuo8f3Epix8SWeF2g2ToITD7xZP4L2scyCT1RoQe_rBo1VAGDo1Jp/s400/av-permission.png" width="400" /></a></div>
<dl style="margin-bottom: 0.5em; margin-top: 0.2em;">Above table is generated by "<i>kernel/goldfish/scripts/selinux/genheaders.c</i>" according to "<i>kernel/goldfish/security/selinux/include/classmap.h</i>". Below figure shows the contents of classmap.h.</dl>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIUcRo9L-vAxloSWdCFTqAkQScYMHEibIurVe3oLHBxq_Y-uh7VPcDSLUQ88EA4EykW3qm0Ts41TO6XxOLbDXwZYKv1uNNOIlCv_FbHnD_GSqVdRP9JBn2eg48jdJXncO1Tx74lQd1MdNW/s1600/classmap.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIUcRo9L-vAxloSWdCFTqAkQScYMHEibIurVe3oLHBxq_Y-uh7VPcDSLUQ88EA4EykW3qm0Ts41TO6XxOLbDXwZYKv1uNNOIlCv_FbHnD_GSqVdRP9JBn2eg48jdJXncO1Tx74lQd1MdNW/s400/classmap.png" width="392" /></a></div>
<dl style="margin-bottom: 0.5em; margin-top: 0.2em;"><br /></dl>
access_vectors is used in <i>external/libsepol/include/sepol/policydb/flask_types.h</i>. It defines the access vector permissions for each class.<br />
<dl style="margin-bottom: 0.5em; margin-top: 0.2em;">Below is an example of <i>access_vectors</i> file</dl>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;"># Define common prefixes for access vectors
#
# common common_name { permission_name ... }
#
# Define a common prefix for file access vectors.
#
common file
{
ioctl
read
write
create
getattr
setattr
lock
relabelfrom
relabelto
append
unlink
link
rename
execute
swapon
quotaon
mounton
}
#
# Define the access vectors.
#
# class class_name [ inherits common_name ] { permission_name ... }
#
# Define the access vector interpretation for file-related objects.
#
class filesystem
{
mount
remount
unmount
getattr
relabelfrom
relabelto
transition
associate
quotamod
quotaget
}
</pre>
security_classes is used in <i>external/libsepol/include/sepol/policydb/flask_types.h</i>. It declares the security classes.<br />
Below is an example of <i>security_classes</i> file
<br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">#Classes marked as userspace are classes
#for userspace object managers
class security
class process
class system
class capability
#file-related classes
class filesystem
class file
class dir
#network-related classes
class socket
class tcp_socket
class udp_socket
</pre>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: inherit;">The relation between access_vectors and security_classes is as below figure:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuylLbXi8znd4hvZ0gRp1TRsNNVGcFD0DyRIfrP5aF0V-B3Bz-538yKeQ7R3p8YMj7mD6dnSUSk32Vv73OaSyvvPqqJIutcwVg5ShBEj05Ut-JZqPtCo6NZC95tGGMQeUrtACnoCAEYWx1/s1600/AV_SC.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuylLbXi8znd4hvZ0gRp1TRsNNVGcFD0DyRIfrP5aF0V-B3Bz-538yKeQ7R3p8YMj7mD6dnSUSk32Vv73OaSyvvPqqJIutcwVg5ShBEj05Ut-JZqPtCo6NZC95tGGMQeUrtACnoCAEYWx1/s400/AV_SC.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnGnoR0IHuz0JhVG6deG93ImAqm6GF38aNzbdgWl5yLNxE-jJulVUAbkCBQRk6bahpH_vciNZwwBkzXeqGIMpTXSAD3PfN88Nn-UutSdLWWDB5Vg8lcT13u3g85LUZJ2Wpq-OZ9rLELWlP/s1600/classmap+binder.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjnGnoR0IHuz0JhVG6deG93ImAqm6GF38aNzbdgWl5yLNxE-jJulVUAbkCBQRk6bahpH_vciNZwwBkzXeqGIMpTXSAD3PfN88Nn-UutSdLWWDB5Vg8lcT13u3g85LUZJ2Wpq-OZ9rLELWlP/s640/classmap+binder.png" width="640" /></a></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">initial_sids, initial_sid_contexts, fs_use, genfs_contexts, port_contexts </span><br />
For flexibility of policy building, these files have been separated to allow additional policy files to be defined for specific devices. <br />
initial_sids is used in <i>external/libselinux/src/avc.c and </i><i>external/libselinux/src/get_initial_context.c</i><i>. </i>It Declares initial SIDs. Such as:<br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">sid kernel
sid security
sid fs
sid file </pre>
initial_sid_contexts declares initial SID contexts. Such as:<br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">sid kernel u:r:kernel:s0
sid security u:object_r:kernel:s0
sid fs u:object_r:labeledfs:s0
sid file u:object_r:unlabeled:s0</pre>
The relation of init_sids and init_sid_contexts is as below figure:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip_1BTsSTA9sZZZiC23BkydNLCyoeaYP33B1tjWdyO9HCDps9BgxKCf0g2RrDRDXOE9tuUDJf-8i6mGLTscuJX8kf4cMKPw9ugTRYmYb6tKNzyqHwlQ8VDRMK9ngJLX3WonAUjbQSt5TxF/s1600/sids.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="381" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip_1BTsSTA9sZZZiC23BkydNLCyoeaYP33B1tjWdyO9HCDps9BgxKCf0g2RrDRDXOE9tuUDJf-8i6mGLTscuJX8kf4cMKPw9ugTRYmYb6tKNzyqHwlQ8VDRMK9ngJLX3WonAUjbQSt5TxF/s400/sids.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKao3jBsZF4EtSF9i5m4TcdLPMf8RcBvUtj8PtDpBTJDoJjuWOW7lJpzwSMLPwsevQeT1fqsnw_0LVjURCP49fhQJtsV9uzIMBLwvsBm8StNYT3lAUf6mzcFxBMpgy2pyZRXAjtheELQDj/s1600/init_sid_contexts.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="345" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKao3jBsZF4EtSF9i5m4TcdLPMf8RcBvUtj8PtDpBTJDoJjuWOW7lJpzwSMLPwsevQeT1fqsnw_0LVjURCP49fhQJtsV9uzIMBLwvsBm8StNYT3lAUf6mzcFxBMpgy2pyZRXAjtheELQDj/s1600/init_sid_contexts.png" width="640" /></a></div>
<br />
fs_use is used in <i>external/libsepol/include/sepol/policydb/service.h</i> & <i>kernel/goldfish/security/selinux/ss/services.c, <b>...</b></i> files.<br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">fs_use_xattr yaff2 u:object_r:labeledfs:s0;
fs_use_xattr ext2 u:object_r:labeledfs:s0;
fs_use_trans tmpfs u:object_r:tmpfs:s0;
fs_use_trans mqueue u:object_r:mqueue:s0</pre>
genfs_contexts is used in <i>kernel/goldfish/security/selinux/hooks.c</i><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">genfscon rootfs / u:object_r:rootfs:s0
# sysfs labels can be set by userspace
genfscon sysfs / u:object_r:sysfs:s0
genfscon vfat / u:object_r:sdcard:s0</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWN6f1_U9SsHYPSrI7WZr61an5aHHLb8PbrmoHRHugnP3iHs2tBZ6_MqFixkV3HddbGrzBPUiHOYJR8Tq8P6NvQCtu2pS50ekVKEAAsUI8yIxwobeT98Y8Cp222VlK_CBO3Jk3PM2-x7pF/s1600/proc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWN6f1_U9SsHYPSrI7WZr61an5aHHLb8PbrmoHRHugnP3iHs2tBZ6_MqFixkV3HddbGrzBPUiHOYJR8Tq8P6NvQCtu2pS50ekVKEAAsUI8yIxwobeT98Y8Cp222VlK_CBO3Jk3PM2-x7pF/s400/proc.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL_-WVM5PBa8wlNXI00MeVUWNZTzK_5VNO0jwK-BEiHYhjnsOgi5DN4yG6ucNhewJD1fIkAUiJ_VrTI0ghv4TihzOnWXUm1o8eYZvI7rxIsjf_6KixoVhD0qJTmzlperh0_y_n3mCZzr3F/s1600/proc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiL_-WVM5PBa8wlNXI00MeVUWNZTzK_5VNO0jwK-BEiHYhjnsOgi5DN4yG6ucNhewJD1fIkAUiJ_VrTI0ghv4TihzOnWXUm1o8eYZvI7rxIsjf_6KixoVhD0qJTmzlperh0_y_n3mCZzr3F/s400/proc.png" width="400" /></a></div>
<br />
<br />
port_contexts defines the socket port contexts. It is used in <i>external/libsepol/src/polcaps.c</i> and <i>kernel/goldfish/security/selinux/selinuxfs.c</i><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;"># portcon statements
portcon tcp 80 u:object_r:http_port:s0</pre>
<br />
<span style="font-family: Courier New, Courier, monospace;">users, roles</span><br />
These define the only user (u) and role (r) used by the policy, although there is no reason why others cannot be added.<br />
users file:<br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">user u roles { r } level s0 range s0 - mls_systemhigh;</pre>
<div>
roles file:</div>
<div>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">role r;
role r types domain;</pre>
</div>
<div>
<br /></div>
<br />
<span style="font-family: Courier New, Courier, monospace;">mls</span><br />
Contains the constraints applied to the defined classes and permissions.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">global_macros, mls_macro, te_marcos</span><br />
These contain the m4 macros that expand the policy files to build a policy in the kernel policy language as described in the <a href="http://selinuxproject.org/page/PolicyLanguage">Policy Language</a> section. The policy can then be compiled by checkpolicy(8). For reference, the policy text file is built and placed in:
<i>out/target/product/<device>/obj/ETC/sepolicy_intermediates/policy.conf</i>
The compiled kernel policy can also be found in this directory and is named sepolicy. There are also policy.conf.dontaudit andsepolicy.dontaudit files that have the dontaudit rules removed. Stripping out these rules stops the auditing of denial messages as they are known events and do not cause any issues. This also helps to manage the audit log by excluding known denial events.<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">attributes </span><br />
Contains the attribute names (forming the <a href="http://selinuxproject.org/page/TypeStatements#attribute_Statement">attribute statements</a>) that will be used to group <a href="http://selinuxproject.org/page/TypeStatements#type_Statement">type identifiers </a>defined by the policy. Types are the basic building blocks for TE rules. SELinux primarily uses types to determine what access is allowed. Attributes and aliases are policy features that ease the management and use of types. We use attributes to refer to a group of types with a single identifier.<br />
The statement definition is:<br />
<pre style="border: 1px dashed rgb(47, 111, 171); padding: 1em;"><span style="background-color: #f9f9f9; font-family: inherit; line-height: 1.1em;">attribute attribute_id;</span>
</pre>
Where:<br />
<table border="1" style="background-color: white; color: black; font-size: 19px; line-height: 28.5625px;"><tbody>
<tr><td><span style="font-size: x-small;">attribute</span></td><td><span style="font-size: x-small;">The attribute keyword<span style="font-family: inherit;">.</span></span></td></tr>
<tr><td><span style="font-size: x-small;">attribute_id</span></td><td><span style="font-size: x-small;">The attribute identifier.</span></td></tr>
</tbody></table>
<br />
<b>Type statement syntax:</b><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">type type_id [alias alias_id] [,attribute_id];</pre>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<b><span style="font-family: inherit;">attributes declarations:</span></b><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;"><span style="font-family: Verdana, sans-serif; font-size: xx-small;">#All types used for devices.
attribute dev_type;
#All types used for processes
attribute domain;
#All types used for domain entry points
attribute exec_type;</span>
</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfbCbXwj7gW9l1Ob1PS4adeaPZhAOZBcz2WjZRv957c1SLPz4TNW855H3p4h3BYfD_p1Ti9OnKj9qoFQJH0zUDlmyuFiFWaXoVA1BDIoyd92B8WErfNbod9FYT8XNlINuPYshDvIyO1KEI/s1600/attribute.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="433" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfbCbXwj7gW9l1Ob1PS4adeaPZhAOZBcz2WjZRv957c1SLPz4TNW855H3p4h3BYfD_p1Ti9OnKj9qoFQJH0zUDlmyuFiFWaXoVA1BDIoyd92B8WErfNbod9FYT8XNlINuPYshDvIyO1KEI/s640/attribute.png" width="640" /></a></div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">policy_capabilities</span><br />
Contains the policy capabilities enabled for the kernel policy (see <a href="http://selinuxproject.org/page/PolicyStatements#policycap_Statement">policycap statement</a>). It is used in <i>external/libsepol/src/polcaps.c</i> and <i>kernel/goldfish/security/selinux/selinuxfs.c</i>.<br />
The statement definition is:<br />
<pre style="border: 1px dashed rgb(47, 111, 171); padding: 1em;"><span style="background-color: #f9f9f9; line-height: 1.1em;">policycap capability;</span>
</pre>
Where:<br />
<table border="1" style="background-color: white; color: black; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;"><tbody>
<tr><td><span style="font-family: Times, Times New Roman, serif; font-size: x-small;">policycap</span></td><td><span style="font-family: Times, Times New Roman, serif; font-size: x-small;">The policycap keyword.</span></td></tr>
<tr><td><span style="font-family: Times, Times New Roman, serif; font-size: x-small;">capability</span></td><td><span style="font-family: Times, Times New Roman, serif; font-size: x-small;">The capability identifier that needs to be enabled for this policy</span><span style="font-family: inherit; font-size: x-small;">.</span></td></tr>
</tbody></table>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;"># This statement enables the network_peer_controls to be enabled
# for use by the policy.
#
policycap network_peer_controls;</pre>
<br />
<span style="font-family: Courier New, Courier, monospace;">*.te</span><br />
The *.te files are the policy module definition files. These are the same format as the standard reference policy and are expanded by the m4 macros. There is (generally) one .te file for each domain/service defined for the device and will contain all the required <a href="http://selinuxproject.org/page/AVCRules">allow</a>, <a href="http://selinuxproject.org/page/TypeRules#Type_Enforcement_Rules">type_transition</a> etc. rules. It will also call any te_macros to access other domains resources (e.g. init_daemon_domain, binder_call).<br />
<br />
<br />
The following files are used to compute and/or configure SE for Android security contexts and reflect information configured within the kernel policy. These files are:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">file_contexts </span><br />
Contains default file contexts for setting the filesystem as standard SELinux. The format of this file is defined in <a href="http://selinuxproject.org/page/PolicyStoreConfigurationFiles#file_contexts_File">file_contexts(5)</a>. The file is installed by default in the root directory. SE for Android services (such as <a href="http://selinuxproject.org/page/NB_SEforAndroid_1#SELinux_Commands">restorecon</a>(8)) will first check for this file at (this is where updated files should be placed):<br />
<i>/data/security/file_contexts</i><br />
If not present they will then check the root directory:<br />
<i>/file_contexts</i><br />
<b>Example file_contexts contents:</b><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">#root
/ u:object_r:rootfs:s0
#Data File
/adb_keys u:object_r:rootfs:s0
/defcult.prop u:object_r:rootfs:s0
#Executables
/init u:object_r:rootfs:s0
/sbin(/.*)? u:object_r:rootfs:s0
#Devices
/dev(/.*)? u:object_r:device:s0
/dev/alarm u:object_r:alarm_device:s0
/dev/audio.* u:object_r:audio_device:s0
</pre>
<div>
<br />
Below figures show the file_contexts you will see in the console.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd0Xh4gDc4gj7IswYVeuITc0NaIY84o6OkLahNgy7XE8gJZKin-riCMcOaRbaxX6z3SGjY3ztK-wHz5QSLxFsPVigWswvR5QLmFi1dSqLQIzYedingeZ7PFwdudxAizwj3J_hfH5f2M4jL/s1600/anr.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="247" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgd0Xh4gDc4gj7IswYVeuITc0NaIY84o6OkLahNgy7XE8gJZKin-riCMcOaRbaxX6z3SGjY3ztK-wHz5QSLxFsPVigWswvR5QLmFi1dSqLQIzYedingeZ7PFwdudxAizwj3J_hfH5f2M4jL/s400/anr.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCKpV8xBPObE27rZR-w1VzdwGfC3TwtNTYe6_BokZMzYxbd6lCq5YDgXmQ07bN3b5MF7iFSUeYBkhzTd8u2Qe8lGfblFZa_41u5JfryyLaBGr_K7eN2OuljP6Q0_tsuviwqQOoXwHZ3lmt/s1600/su.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="210" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCKpV8xBPObE27rZR-w1VzdwGfC3TwtNTYe6_BokZMzYxbd6lCq5YDgXmQ07bN3b5MF7iFSUeYBkhzTd8u2Qe8lGfblFZa_41u5JfryyLaBGr_K7eN2OuljP6Q0_tsuviwqQOoXwHZ3lmt/s400/su.png" width="400" /></a></div>
<br />
<br /></div>
<span style="font-family: Courier New, Courier, monospace;">property_contexts</span><br />
Contains default contexts to be applied to Android property services as discussed in the <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#property_contexts_File">property_contexts file</a> section. The file is installed by default in the root directory. The SE for Android initialisation / reload process will first check for this file at (this is where updated files should be placed):<br />
<i>/data/security/property_contexts</i><br />
If not present they will then check the root directory:<br />
<i>/property_contexts</i><br />
This file is used in <i>system/core/init/initc.</i><br />
<div>
<b>Example of property_contexts</b></div>
<div>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">##########################
# property service keys
#
net.rmnet0 u:object_r:radio_prop:s0
net.gprs u:object_r:radio_prop:s0
net.ppp u:object_r:radio_prop:s0
net.qmi u:object_r:radio_prop:s0
net.lte u:object_r:radio_prop:s0
net.cdma u:object_r:radio_prop:s0
gsm. u:object_r:radio_prop:s0
persist.radio u:object_r:radio_prop:s0
net.dns u:object_r:radio_prop:s0
sys. u:object_r:system_prop:s0
service. u:object_r:system_prop:s0
wlan. u:object_r:system_prop:s0
dhcp. u:object_r:system_prop:s0
debug. u:object_r:shell_prop:s0
log. u:object_r:shell_prop:s0</pre>
</div>
<br />
system/core/init/property_service.c will set property services<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLIQalj-PeLMel1bAigcqcbmoc5fBBn7AzCM0iVff6TGt_mAM4kRwc7kJbze335c0kfohTByurureJHBjSLt3tHHdqtTezDSPL_Av0JuOc4SERvQGL67G4UWIe0faNtdz0aW5GVT_DHkZU/s1600/property_service.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="500" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLIQalj-PeLMel1bAigcqcbmoc5fBBn7AzCM0iVff6TGt_mAM4kRwc7kJbze335c0kfohTByurureJHBjSLt3tHHdqtTezDSPL_Av0JuOc4SERvQGL67G4UWIe0faNtdz0aW5GVT_DHkZU/s1600/property_service.png" width="640" /></a></div>
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">seapp_contexts </span><br />
Contains information to allow domain or file contexts to be computed based on parameters as discussed in the <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#seapp_contexts_File">seapp_contexts file</a> section. The file is installed by default in the root directory. The SE for Android initialisation / reload process will first check for this file at (this is where updated files should be placed):<br />
<i>/data/security/seapp_contexts</i><br />
If not present they will then check the root directory:<br />
<i>/seapp_contexts</i><br />
This file is used in <i>external/sepolicy/check_seapp/check_seapp.c</i> and <i>external/libselinux/src/android.c</i><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">isSystemServer=true domain=system
user=system domain=system_app type=system_data_file
user=bluetooth domain=bluetooth type=bluetooth_data_file
user=nfc domain=nfc type=nfc_data_file
user=radio domain=radio type=radio_data_file
user=_app domain=untrusted_app type=app_data_file levelFrom=app
user=_app seinfo=platform domain=platform_app type=platform_app_data_file
user=_app seinfo=shared domain=shared_app type=platform_app_data_file
user=_app seinfo=media domain=media_app type=platform_app_data_file
user=_app seinfo=release domain=release_app type=platform_app_data_file
user=_isolated domain=isolated_app</pre>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd3CCkq5PgbQBPoxS8A40alghR0t_H4ub7BudFhoHiLtXPpC69jnec6mcB9BBC1b-pKRapxmpeqj7tTGhVhDPm_0uDIXY1rjqrcZ0bdovQEU-fyxiKBcLbuekP60LZBCZzZXR_6Nyr769P/s1600/seappcontext-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="330" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd3CCkq5PgbQBPoxS8A40alghR0t_H4ub7BudFhoHiLtXPpC69jnec6mcB9BBC1b-pKRapxmpeqj7tTGhVhDPm_0uDIXY1rjqrcZ0bdovQEU-fyxiKBcLbuekP60LZBCZzZXR_6Nyr769P/s640/seappcontext-2.png" width="640" /></a></div>
<br />
Above picture shows "<i>untrusted_app</i>" domain defined in seapp_contexts and the process contexts shown in system.<br />
If the "<i>untrusted_app</i>" domain is not defined in seapp_coontexts, the process context will become "<i><span style="color: blue;">zygote</span></i>", a default label. The picture is shown as below.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEc_l4iZGs6czlYWn4hli5V4y5csWHt_97sgBHv3BA42lYNRyrISmEKVNts5cwflw9oUt6NOFLRd4rOK1nWyy0lUhk4qm5Bcp73Yjsx-HLViN9zmQ52oM5KYcz4jPSf8Msnxir_HTpFOQ0/s1600/seappcontext-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEc_l4iZGs6czlYWn4hli5V4y5csWHt_97sgBHv3BA42lYNRyrISmEKVNts5cwflw9oUt6NOFLRd4rOK1nWyy0lUhk4qm5Bcp73Yjsx-HLViN9zmQ52oM5KYcz4jPSf8Msnxir_HTpFOQ0/s640/seappcontext-1.png" width="640" /></a></div>
<br />
<span style="font-family: Courier New, Courier, monospace;">selinux-network.sh </span><br />
If using iptables(8) then SECMARK information may be configured in this file as part of the build. It is installed in system/bin and executed at system initialisation time.
<br />
<br />
<br /></dd><dd style="margin-bottom: 0.1em; margin-left: 2em;"></dd><span style="font-family: sans-serif;"><span style="background-color: white; font-size: 19px; line-height: 1.5em;">Reference: </span></span></dl>
<div style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 1.5em;">
<a href="http://selinuxproject.org/page/NB_SEforAndroid_1#SELinux_MAC_Policy_Files">http://selinuxproject.org/page/NB_SEforAndroid_1#SELinux_MAC_Policy_Files</a><br />
<a href="http://selinuxproject.org/page/NB_MLS">http://selinuxproject.org/page/NB_MLS</a><br />
<a href="http://selinuxproject.org/page/MLSStatements">http://selinuxproject.org/page/MLSStatements</a></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-30829944458847716062013-05-24T01:52:00.000-07:002013-06-03T02:18:55.637-07:00SEforAndroid<h3>
Policy</h3>
1. Policy sources are located at "<i>external/sepolicy</i><span style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;">". </span>There may also be policy configuration files to enable specific device features under <i>device/<vendor>/<device>/sepolicy</i> directories <br />
<br />
2. The policy consists of source files used to generate the SELinux kernel policy file, a file_contexts configuration, a property_contexts configuration, a seapp_contexts configuration, and a mac_permissions.xml configuration.<br />
<ul>
<li>The <span style="color: blue;">file_contexts</span> configuration is used to label files at build time (e.g. the system partition) and at runtime (e.g. device nodes, service socket files, /data directories created by init.rc, ...). The <span style="color: blue;">file_contexts</span> configuration content of AOSP and SEforAndroid are a little different. It will be used by <i>external/libselinux/src/android.c</i> and <i>external/libsepol/src/module.c</i>.</li>
<li>The <span style="color: blue;">property_contexts</span> configuration is used to specify the security context of Android properties for permission checking purposes. The <span style="color: blue;">property_contexts</span> configuration content of AOSP and SEforAndroid are a little different. It will be used by <i>system/core/init/init.c</i> and <i>frameworks/base/services/java/com/android/server/DevicePolicyManagerService.java</i>.</li>
<li>The <span style="color: blue;">seapp_contexts</span> configuration is used to label app processes and app package directories. The <span style="color: blue;">seapp_contexts</span> configuration content of AOSP and SEforAndroid are a little different. It is used by <i>external/libselinux/src/android.c</i> and <i>frameworks/base/services/java/com/android/server/DevicePolicyManagerService.java</i>.</li>
<li>The <span style="color: blue;">mac_permissions.xml</span> configuration is the middleware MAC policy. The <span style="color: blue;">mac_permissions.xml</span> configuration content of AOSP and SEforAndroid are a little different. It is used by <i>frameworks/base/services/java/com/android/server/DevicePolicyManagerService.java, external/sepolicy/tools/setool/src/com/seandroid/tools/PolicyParser.java, </i><i>frameworks/base/services/java/com/android/server/pm/SELinuxMMAC.java </i>and some <i>*.py files.</i></li>
<li>The property_contexts, seapp_contexts, and mac_permissions.xml configurations are unique to SE for Android (i.e. <span style="background-color: yellow;">they were not part of the regular SELinux policy</span>).</li>
</ul>
<div>
<br /></div>
<div>
3.Device-specific policy can be specified by defining <span style="color: blue;">BOARD_SEPOLICY_DIRS</span>, <span style="color: blue;">BOARD_SEPOLICY_UNION</span> and/or <span style="color: blue;">BOARD_SEPOLICY_REPLACE, BOARD_SEPOLICY_IGNORE</span> variables in a<b> BoardConfig.mk</b> file under the device/<vendor>/<device> or vendor/<vendor>/<device> directories.</div>
<div>
<ul>
<li><b>BOARD_SEPOLICY_UNION</b> is a list of files that will be "unioned", IE concatenated, at the END of their respective file in external/sepolicy. Note, to add a unique file you would use this variable.</li>
<li><b>BOARD_SEPOLICY_REPLACE</b> is a list of files that will be used instead of the corresponding file in external/sepolicy.</li>
<li><b>BOARD_SEPOLICY_DIRS</b> contains a list of directories to search for BOARD_SEPOLICY_UNION and BOARD_SEPOLICY_REPLACE files. Order matters in this list.</li>
<li><b>BOARD_SEPOLICY_IGNORE</b> is a list of paths (directory + filename) of files that are not to be included in the resulting policy. This list is passed to filter-out to remove any paths you may want to ignore. This is useful if you have numerous config directories that contain a file and you want to NOT include a particular file in your resulting policy file, either by UNION or REPLACE.</li>
</ul>
</div>
<div>
4. SELinux kernel policy is presently compiled as part of the Android build and added to the ramdisk image so that it can be loaded by init very early in boot, before mounting the system partition.</div>
<div>
<br /></div>
<div>
5.Once the data partition has been mounted, policy can be reloaded from <b>/data/security</b> by placing policy files under /data/security and setting the selinux.reload_policy property to 1 (<span style="color: blue;">setprop selinux.reload_policy 1</span>). This will trigger a reload of policy by init, which will also restart ueventd and installd so that they can reload the policy configuration files relevant to their operation.</div>
<div>
<br /></div>
<div>
6.The initialisation or policy reload process will always check for a policy at /data/security/sepolicy first and then if not present at /sepolicy. </div>
<div>
<span style="background-color: white;"><br /></span></div>
<div>
<span style="background-color: white;">7.</span>The policy can then be compiled by checkpolicy(8). For reference, the policy text file is built and placed in: </div>
<div>
out/target/product/<device>/obj/ETC/sepolicy_intermediates/policy.conf</div>
<div>
<br /></div>
<div>
8. The *.te files are the policy module definition files. These are the same format as the standard reference policy and are expanded by the m4 macros. There is (generally) one .te file for each domain/service defined for the device and will contain all the required <a href="http://selinuxproject.org/page/AVCRules">allow</a>, <a href="http://selinuxproject.org/page/TypeRules#Type_Enforcement_Rules">type_transition</a> etc. rules. It will also call any te_macros to access other domains resources (e.g. init_daemon_domain, binder_call).</div>
<div>
<span style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;"><br /></span></div>
<div>
9. <b>seapp_contexts</b> --</div>
This file is loaded and sorted into memory automatically on first use of one of the following SE for Android<i> libselinux</i> functions that are called by the SE for Android enabled services:<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">selinux_android_setcontext </span>- Computes process security <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">contexts.selinux_android_setfilecon2</span> - Computes file/directory security contexts.<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">selinux_android_seapp_context_reload</span> will also reload this file.<br />
<div>
<br />
Above functions are implemented in <i style="background-color: blue;"><span style="color: white;">external/libselinux/src/android.c</span></i><br />
<br />
Input selectors from <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">seapp_contexts</span> file:</div>
<div>
<ul>
<li>isSystemServer (boolean)</li>
<li>user (string)</li>
<li>seinfo (string)</li>
<li>name (string) - A package name e.g. com.example.demo</li>
<li>sebool (string) - The boolean must be ‘active’ (enabled/true)</li>
</ul>
</div>
<div>
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">isSystemServer=true</span> can only be used once. An unspecified <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">isSystemServer</span> defaults to false.<br />
<br />
An unspecified string selector will match any value.<br />
<br />
A user string selector that ends in * will perform a prefix match.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">user=app_*</span> will match any regular app UID.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">user=isolated</span> will match any isolated service UID.<br />
<br />
All specified input selectors in an entry must match (i.e. logical AND).<br />
<br />
Matching is case-insensitive.<br />
<br />
<b>Precedence rules:</b></div>
<div>
1) isSystemServer=true before isSystemServer=false.</div>
<div>
2) Specified user= string before unspecified user= string.</div>
<div>
3) Fixed user= string before user= prefix (i.e. ending in *).</div>
<div>
4) Longer user= prefix before shorter user= prefix.</div>
<div>
5) Specified seinfo= string before unspecified seinfo= string.</div>
<div>
6) Specified name= string before unspecified name= string.</div>
<div>
7) Specified sebool= string before unspecified sebool= string.<br />
<br />
<b>Outputs:</b></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">domain (string)</span> - The type component of a process context.</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">type (string)</span> - The type component of a file/directory context.level</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">From (string; one of none, all, app, or user)</span> - A level that will be automatically computed based on the parameter.</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">level (string) </span>- A predefined level (e.g. s0:c1022.c1023)<br />
<br />
Only entries that specify <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">domain=</span> will be used for app process labeling.<br />
<br />
Only entries that specify <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">type=</span> will be used for app directory labeling.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">levelFrom=user</span> is only supported for _app or _isolated UIDs.<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">levelFrom=app</span> or <span style="font-family: Courier New, Courier, monospace; font-size: x-small;">levelFrom=all</span> is only supported for _app UIDs.<br />
<br />
level may be used to specify a fixed level for any UID.<br />
<br /></div>
<div>
10. <b>file_contexts --</b></div>
Contains default file contexts for setting the filesystem as standard SELinux. The format of this file is defined in <a href="http://selinuxproject.org/page/PolicyStoreConfigurationFiles#file_contexts_File">file_contexts</a>. The file is installed by default in the root directory. SE for Android services (such as <a href="http://selinuxproject.org/page/NB_SEforAndroid_1#SELinux_Commands">restorecon</a>) will first check for this file at (this is where updated files should be placed): <span style="background-color: white; font-family: monospace; font-size: 16px; line-height: 28.5625px;">/data/security/file_contexts</span><br />
<span style="background-color: white; font-family: monospace; font-size: 16px; line-height: 28.5625px;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs7fCf6X9HgyAUblo3x2_SPGB2t8r2BWBRCpfEDqaqoqMMTK8rTRH5D1iUwe8Z9bxTcKLo4wpx8YHcarJmfFZdzG0djwh9ywmzUtx78baF-5tx-_ATCbpQK_V6ahpz1lsMvFVcD6gVolGd/s1600/file_contexts.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs7fCf6X9HgyAUblo3x2_SPGB2t8r2BWBRCpfEDqaqoqMMTK8rTRH5D1iUwe8Z9bxTcKLo4wpx8YHcarJmfFZdzG0djwh9ywmzUtx78baF-5tx-_ATCbpQK_V6ahpz1lsMvFVcD6gVolGd/s1600/file_contexts.png" height="416" width="640" /></a></div>
<span style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;"><br /></span>
This file becomes the policies<i> ./contexts/files/file_contexts</i> file and is built from entries in the<i> ./modules/active/file_contexts.template</i> file as explained above and shown in above picture. It is then used by the file labeling utilities to ensure that files and directories are labeled according to the policy.<br />
<br />
<b>Example file_contexts contents:</b><br />
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">#root
/ u:object_r:rootfs:s0
#Data File
/adb_keys u:object_r:rootfs:s0
/defcult.prop u:object_r:rootfs:s0
#Executables
/init u:object_r:rootfs:s0
/sbin(/.*)? u:object_r:rootfs:s0
#Devices
/dev(/.*)? u:object_r:device:s0
/dev/alarm u:object_r:alarm_device:s0
/dev/audio.* u:object_r:audio_device:s0
</pre>
<br />
11.<b> property_contexts --</b><br />
<div>
Contains default contexts to be applied to Android property services as discussed in the <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#property_contexts_File">property_contexts file</a> section. The file is installed by default in the root directory. The SE for Android initialisation / reload process will first check for this file at (this is where updated files should be placed):<br />
<div>
/data/security/property_contexts<br />
<div>
<br /></div>
This file holds property names and their contexts that will be applied by SELinux when applications are loaded. The property names reflect the 'white list' of Android property entries that are also built into the system (see <i>system/core/init/property_service.c</i> and<i> init.c</i>) however there are also additional property entries for applications that require specific contexts to be set.</div>
<div>
<br /></div>
<div>
Each line within the <b>property contexts</b> file is as follows:</div>
<div>
<span style="background-color: blue;"><span style="color: #f3f3f3;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">property_key context
</span></span></span><br />
<span style="background-color: blue;"><span style="color: #f3f3f3;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></span>
Where:</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">property_key</span> The key used to obtain the context that may contain '*' for wildcard matching.</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">context</span> The security context that will be applied to the object.</div>
<div>
<br /></div>
<div>
<b>Example of property_contexts</b></div>
<div>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">##########################
# property service keys
#
net.rmnet0 u:object_r:radio_prop:s0
net.gprs u:object_r:radio_prop:s0
net.ppp u:object_r:radio_prop:s0
net.qmi u:object_r:radio_prop:s0
net.lte u:object_r:radio_prop:s0
net.cdma u:object_r:radio_prop:s0
gsm. u:object_r:radio_prop:s0
persist.radio u:object_r:radio_prop:s0
net.dns u:object_r:radio_prop:s0
sys. u:object_r:system_prop:s0
service. u:object_r:system_prop:s0
wlan. u:object_r:system_prop:s0
dhcp. u:object_r:system_prop:s0
debug. u:object_r:shell_prop:s0
log. u:object_r:shell_prop:s0
</pre>
<h4>
Build Tools</h4>
The kernel policy is compiled using checkpolicy(8) via the external/sepolicy/Android.mk file. There are also a number of SE for Android specific tools used to assist in policy configuration that are described in <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#Policy_Build_Tools">Policy Build Tools</a>, with a summary as follows:</div>
<div>
<ul>
<li><a href="http://selinuxproject.org/page/NB_SEforAndroid_2#checkfc">checkfc</a> - Used to parse the file_contexts file against the binary policy sepolicy. This is to ensure all file contexts are valid for the policy. There is a -p option that is used to validate the contexts defined in the property_contexts file. Example validating file_contexts file:<pre style="border: 1px dashed rgb(47, 111, 171); padding: 1em;"><span style="background-color: #f9f9f9; line-height: 1.1em;">checkfc out/target/product/generic/root/sepolicy out/target/product/generic/root/file_contexts</span>
</pre>
</li>
<li><br /></li>
Example validating property_contexts file:<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">checkfc -p out/target/product/generic/root/sepolicy out/target/product/generic/root/property_contexts</pre>
<li><a href="http://selinuxproject.org/page/NB_SEforAndroid_2#checkseapp">checkseapp</a> - Used to parse the seapp_contexts file against the binary policy sepolicy. This is to ensure policy related information is valid (e.g. checks booleans, types, levels etc.).</li>
<li><a href="http://selinuxproject.org/page/NB_SEforAndroid_2#insertkeys.py">insertkeys.py</a> - Used to replace keywords in the signature sections of the mac_permissions.xml and mmac_types.xml files with keys obtained frompem files. This uses information contained in the external/sepolicy/keys.conf file that is detailed in the <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#keys.conf">keys.conf</a> section. This utility will also strip files of comments.</li>
<li><a href="http://selinuxproject.org/page/NB_SEforAndroid_2#setool">setool</a> - This is not used by the actual build process but assists in generating new entries for the mac_permissions.xml file. It will extract permissions from one or more packages with their signatures then generate the package sections. Its output may need to be modified before inclusion in the master file as detailed in the <a href="http://selinuxproject.org/page/NB_SEforAndroid_2#setool">setool</a> section.</li>
</ul>
</div>
<div>
<br />
<h4>
Modifying and Reloading Policy</h4>
<div>
This is covered at <a href="http://selinuxproject.org/page/SEforAndroid#Policy">SEforAndroid - Policy</a> in detail. This section gives a brief overview:<br />
<ul>
<li style="margin-bottom: 0.1em;">Modify the required policy source files, then regenerate the kernel policy file by:</li>
</ul>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">make sepolicy
</pre>
<ul>
<li style="margin-bottom: 0.1em;">Copy the policy file to the device:</li>
</ul>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">adb push out/target/product/<device>/root/sepolicy /data/security
</pre>
<ul>
<li style="margin-bottom: 0.1em;">Then load the new policy by:</li>
</ul>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">adb shell su 0 setprop selinux.reload_policy 1</pre>
<div>
<div>
<br /></div>
<h3>
Middleware MAC</h3>
<ul>
<li>Install-time MAC (merged to seandroid),</li>
<li>Permission revocation (revoke-perms),</li>
<li>Intent MAC (intent_mac).</li>
</ul>
<div>
<br /></div>
<h4>
Install-time MAC</h4>
<div>
1. This mechanism applies an install-time check of app permissions against a MAC policy configuration (found in <b><i><span style="color: blue;">external/sepolicy/mac_permissions.xml</span></i></b> in the source tree and as <i><span style="color: blue;"><b>/system/</b></span></i><b><i><span style="color: blue;">etc/security/mac_permissions.xml</span></i></b> on the system image).</div>
<div>
<br /></div>
<div>
2. The main code for the service is </div>
<div>
frameworks/base/services/java/com/android/server/pm/SELinuxMMAC.java</div>
<div>
<tt style="background-color: white; line-height: 28.5625px;"><br /></tt></div>
<div>
3. The persist.mmac.enforce system property controls whether the MAC restrictions are enforced; this can be set via SEAdmin app or via setprop.</div>
<div>
<br /></div>
<div>
4. The setool program can be used to generate policy stanzas for mac_permissions.xml, or to check whether a given apk would violate a given mac_permissions.xml configuration.</div>
<div>
<pre style="background-color: #f9f9f9; border: 1px dashed rgb(47, 111, 171); line-height: 1.1em; padding: 1em;">setool --build whitelist /path/to/foo.apk
setool --policy /path/to/mac_permissions.xml /path/to/foo.apk</pre>
</div>
<div>
<br /></div>
<div>
5. Install-time MMAC policy that checks whether app permissions are allowed or not. If not allowed, the app cannot be installed or if the app is already installed before the updated policy, then the app cannot be run after the update.</div>
<div>
<br /></div>
<h4>
Premission Revocation</h4>
<div>
1. A revoked permission list is maintained for each package and checked at runtime on permission checks. </div>
<div>
<br /></div>
<div>
2. A revoked permissions configuration (found in <i><span style="color: blue;">external/mac-policy/revoke_permissions.xml</span></i> in the source tree and as <i><span style="color: blue;">etc/security/revoke_permissions.xml</span></i> on the system image) can specify revocation lists that are applied automatically on each boot.</div>
<div>
<br /></div>
<div>
3. Revoke permissions policy that checks whether the policy configured permissions on a package basis will be revoked at run time. If not allowed, that permission will be revoked (i.e. allow all unless specifically revoked). This is an optional policy and does not require any specific SELinux policy support.</div>
<div>
<br /></div>
<div>
4. The file that configures the policy is the revoke_permissions.xml and by default is installed at:</div>
<div>
/system/etc/security/revoke_permissions.xml</div>
<div>
<br /></div>
<div>
5. The main code for the service is self contained in:</div>
<div>
frameworks/base/services/java/com/android/server/pm/PackageManagerService.java</div>
<h4>
Intent MAC</h4>
<div>
1. Intent MAC supports a white-list of Intents and the types of their sources and destinations.</div>
<div>
<br /></div>
<div>
2.<span style="background-color: white; font-family: sans-serif; font-size: 19px; line-height: 28.5625px;"> </span>Currently, we only protect delivery of Intents to Activities, Broadcast Receivers, and Services. Content Providers are not yet protected.</div>
<div>
<br /></div>
<div>
3. Intent MMAC policy that checks whether the policy configured app intents (on a package and/or signature basis) are allowed or not at run time. If not allowed, that intent will not be sent. </div>
<div>
<br /></div>
<div>
4. The files that configure policy are intent_mac.xml and mmac_types.xml and by default they are installed at: </div>
<div>
/system/etc/security/intent_mac.xml</div>
<div>
/system/etc/security/mmac_types.xml</div>
<div>
<br /></div>
<div>
5. The main code for the service is <br />
frameworks/base/core/java/andriod/content/pm/IntentMAC.java and MMACtypes.java</div>
<div>
<br /></div>
<br />
<h4>
auditd Daemon</h4>
<br />
The majority of this text has been extracted from system/core/auditd/README that describes its configuration, however note that:<br />
<ul>
<li>Kernel auditing is included by default when building the SE for Android supplied kernels (e.g. for goldfish kernel see thekernel/goldfish/arch/arm/configs/goldfish_arm7_defconfig file) and auditd daemon is also included by default (seebuild/target/product/core.mk - under the SELinux packages).</li>
<li>The audit daemon output is formatted so that ausearch(8) can be used to search for SELinux events.</li>
</ul>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2s2OIAxUFsDre118W7vwBicXWjAccy0X0gOgAcJf8W8VoRMs8wcN6lb9lonJrmMSnpNqGnvO14dvokcJfBbnKvGQtJ-MjqwSs9bmF584Yr3gdB4x_BAlA_IyU-OX4Jb2DFE-R3Qi2UF-g/s1600/selinux+architecture.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2s2OIAxUFsDre118W7vwBicXWjAccy0X0gOgAcJf8W8VoRMs8wcN6lb9lonJrmMSnpNqGnvO14dvokcJfBbnKvGQtJ-MjqwSs9bmF584Yr3gdB4x_BAlA_IyU-OX4Jb2DFE-R3Qi2UF-g/s1600/selinux+architecture.png" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
reference:</div>
<div>
1. <a href="http://selinuxproject.org/page/SEforAndroid#Middleware_MAC">http://selinuxproject.org/page/SEforAndroid</a></div>
<div>
2. <a href="http://selinuxproject.org/page/NB_SEforAndroid_1#Supported_MAC_Services">http://selinuxproject.org/page/NB_SEforAndroid_1</a></div>
<div>
3. <a href="http://selinuxproject.org/page/NB_SEforAndroid_1">http://selinuxproject.org/page/NB_SEforAndroid_</a>2</div>
</div>
<div>
4.<a href="http://selinuxproject.org/page/PolicyStoreConfigurationFiles">http://selinuxproject.org/page/PolicyStoreConfigurationFiles</a></div>
</div>
</div>
<div>
5. <a href="http://selinuxproject.org/page/NB_PolicyType#Policy_Versions">http://selinuxproject.org/page/NB_PolicyType</a></div>
</div>
<div>
<a href="http://selinuxproject.org/page/NB_RefPolicy">http://selinuxproject.org/page/NB_RefPolicy</a></div>
<div>
<a href="http://selinuxproject.org/page/PolicyStatements">http://selinuxproject.org/page/PolicyStatements</a></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com1tag:blogger.com,1999:blog-7057734095102577753.post-69351970785491202292013-05-23T00:40:00.000-07:002013-06-27T00:57:52.878-07:00Android SDK Commands<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># Android模擬器命令列啟動模式</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">在android-sdk-windows-1.1\tools執行emulator以執行模擬器</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">加上-skin參數,指定顯示模式為HVGA-L,則可轉為橫向</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - skin HVGA-L (480*320,水平顯示)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - skin HVGA-L (320*480,垂直顯示,模擬器預設模式)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - skin HVGA-L (320*240,水平顯示)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - skin HVGA-L (240*320,垂直顯示)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 使用mksdcard指令模擬1GB的記憶卡</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">mksdcard 1024M sdcard.img</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 模擬插入 SD 卡的模擬器</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - sdcard sdcard.img</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 使用 adb+push 上載檔案到SD記憶卡</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb push 001.jpg /sdcard (複製檔案到 /sdcard 目錄下)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb push pictures /sdcard (複製 picture 照片目錄到 /sdcard 目錄下)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb push mp3 /sdcard (複製 mp3 音樂目錄到 /sdcard 目錄下)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb shell (Android 模擬器啟動命令列模式)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">#cd /sdcard (進入 /sdcard 目錄)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">#ls (查看 SD 記憶卡中的檔案)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"><span style="color: red;"># 使用 adb+pull 從 SD 記憶卡下載檔案</span></span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb pull /sdcard/001.jpg . (下載 /sdcard 目錄下的檔案)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb pull /sdcard/pictures . (下載 sdcard 目錄下的 pictures 目錄)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 刪除 SD 卡裡面的檔案</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb shell</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">#ced /sdcard</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">#rm 001.jpg (刪除 SD 記憶卡裡的檔案)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">#rm -r * (刪除 SD 記憶卡裡所有檔案與目錄)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># Android模擬器影片播放方法</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">mksdcard 4096M video.img (製作一個影像檔的 SD 記憶卡)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb push video.avi /sdcard (從電腦複製影像檔到 SD 卡中)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -sdcard video.img (啟動模擬器並載入 SD 卡)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">下載免費的影片播放軟體,ex: Meridian Video Player (iiivpa.apk)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">http://sites.google.com/site/eternalsandbox/Home/meridian-video-player</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb install iiivpa.apk (安裝Meridian Video Player)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">接下來就可以用裝上去的player播放.mp4、3gp與.wmv三種檔案格式</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 安裝 APK 應用程式</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb install filename.apk (安裝filename.apk)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb install -r filename.apk (保留已設定資料,重新安裝filename.apk)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb -s emulator-5554 install filename.apk (指定安裝 APK 套件在 5554 的 Android 模擬器中)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># 移除 APK 應用程式</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb uninstall package</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb uninstall -k package (移除程式時,保留資料)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">此package名稱不是安裝APK套裝時的檔名或顯示在模擬器中的應用程式名稱</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">可以先到/data/data或data/app目錄下,查詢想移除的package名稱</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb shell</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">ls /data/data 或 /data/app (查詢 Package 名稱)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">exit</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb uninstall package (移除查詢到的 Package)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"># ADB 系統除錯與連結工具</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb devices (顯示目前有多少個模擬器正在執行)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb -s <serialNumber> <command> (指定模擬器來操作)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb -s emulator-5554 install email.apk</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb install apkfile (安裝 APK 應用程式套件)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb install email.apk</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb uninstall package (移除 APK 應用程式套件)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb uninstall com.android.email</span><br />
<span style="background-color: white; color: red; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb shell (進入 Android 系統指令列模式)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$ls</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$dmesg (查看 Android Linux Kernel 運作訊息)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">ls - 顯示檔案目錄</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">cd - 進入目錄</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">rm - 刪除檔案</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">mv - 移動檔案</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">mkdir - 產生目錄</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">rmdir - 刪除目錄</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb push <file/dir> (複製檔案到 SD 卡)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb push mp3 /sdcard</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb pull <file/dir> . (從 Android 系統下載檔案)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">adb pull /data/app/com.android.email</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb logcat (監控模擬器運作紀錄,以Ctrl + c 離開監控模式)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb bugreport (產生 adb 除錯報告)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb get-state (獲得 adb 伺服器運作狀態)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb start-server (啟動 adb 伺服器)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb kill-server (關掉 adb 伺服器)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb forward tcp:6100 tcp:7100 (更改模擬器網路 TCP 通訊埠)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb shell ps -x (顯示 Android 上所有正在執行的行程)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb version (顯示 adb 版本)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">$adb help (顯示 adb 指令參數)</span><br />
<br style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;" />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"><span style="color: red;"># Emulator 命令列啟動參數</span></span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -timezone Asia/Taipei (指定時區)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -no-boo-anim (省略開機小機器人動畫畫面)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -scale auto (調整模擬器視窗大小)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator - scale factor (factor: 0.1-3.0)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -dpi-device 300 (更改模擬器的解析度,default為 165dpi)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -skin <skinID> (更改模擬器顯示模式)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -help-keys (顯示鍵盤快速鍵說明)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -shell (相當於adb shell 功能)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -data data.img (使 /data 目錄使用 data.img 的檔案空間)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -sdcard sdcard.img (使 /sdcard 目錄使用 sdcard.img 的檔案空間)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -cache cache.img (瀏覽器暫存檔儲存空間)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -wipe-data (使模擬器恢復到原廠設定)</span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;">emulator -help (顯示 emulator 指令參數) </span><br />
<span style="background-color: white; color: #777777; font-family: Arial, sans-serif; line-height: 15.199999809265137px;"><br /></span>
<br />
<span style="color: red;">#Android : adb shell am 命令列啟動Activity或傳送Intent</span><br /><br />例如:<br />//啟動瀏覽器,打開目標網址 <br />adb shell am -a android.intent.action.VIEW -d http://www.google.com<br />adb shell am broadcast -a org.meshpoint.anode.START -e cmdline /data/tmp/hello.js<br />//撥打電話,號碼是123456789<br />adb shell am start -a android.intent.action.CALL -d tel:123456789<br />am start -a android.intent.action.CALL -d tel:123456789<br />Starting: Intent { action=android.intent.action.CALL data=tel:123456789}<br />
<div>
<br /></div>
<div>
輸入按鍵 : adb shell input keyevent [KEY ID]<br /> 例如,想要「解鎖」就鍵入「adb shell input keyevent 82」<br /> 00 -> "KEYCODE_UNKNOWN"<br /> 01 -> "KEYCODE_MENU"<br /> 02 -> "KEYCODE_SOFT_RIGHT"<br /> 03 -> "KEYCODE_HOME"<br /> 04 -> "KEYCODE_BACK"<br /> 05 -> "KEYCODE_CALL"<br /> 06 -> "KEYCODE_ENDCALL"<br /> 07 -> "KEYCODE_0"<br /> 16 -> "KEYCODE_9"<br /> 17 -> "KEYCODE_STAR"<br /> 18 -> "KEYCODE_POUND"<br /> 19 -> "KEYCODE_DPAD_UP"<br /> 20 -> "KEYCODE_DPAD_DOWN"<br /> 21 -> "KEYCODE_DPAD_LEFT"<br /> 22 -> "KEYCODE_DPAD_RIGHT"<br /> 23 -> "KEYCODE_DPAD_CENTER"<br /> 24 -> "KEYCODE_VOLUME_UP"<br /> 25 -> "KEYCODE_VOLUME_DOWN"<br /> 26 -> "KEYCODE_POWER"<br /> 27 -> "KEYCODE_CAMERA"<br /> 28 -> "KEYCODE_CLEAR"<br /> 29 -> "KEYCODE_A"<br /> 54 -> "KEYCODE_Z"<br /> 55 -> "KEYCODE_COMMA"<br /> 56 -> "KEYCODE_PERIOD"<br /> 57 -> "KEYCODE_ALT_LEFT"<br /> 58 -> "KEYCODE_ALT_RIGHT"<br /> 59 -> "KEYCODE_SHIFT_LEFT"<br /> 60 -> "KEYCODE_SHIFT_RIGHT"<br /> 61 -> "KEYCODE_TAB"<br /> 62 -> "KEYCODE_SPACE"<br /> 63 -> "KEYCODE_SYM"<br /> 64 -> "KEYCODE_EXPLORER"<br /> 65 -> "KEYCODE_ENVELOPE"<br /> 66 -> "KEYCODE_ENTER"<br /> 67 -> "KEYCODE_DEL"<br /> 68 -> "KEYCODE_GRAVE"<br /> 69 -> "KEYCODE_MINUS"<br /> 70 -> "KEYCODE_EQUALS"<br /> 71 -> "KEYCODE_LEFT_BRACKET"<br /> 72 -> "KEYCODE_RIGHT_BRACKET"<br /> 73 -> "KEYCODE_BACKSLASH"<br /> 74 -> "KEYCODE_SEMICOLON"<br /> 75 -> "KEYCODE_APOSTROPHE"<br /> 76 -> "KEYCODE_SLASH"<br /> 77 -> "KEYCODE_AT"<br /> 78 -> "KEYCODE_NUM"<br /> 79 -> "KEYCODE_HEADSETHOOK"<br /> 80 -> "KEYCODE_FOCUS"<br /> 81 -> "KEYCODE_PLUS"<br /> 82 -> "KEYCODE_MENU"<br /> 83 -> "KEYCODE_NOTIFICATION"<br /> 84 -> "KEYCODE_SEARCH"<br /> 85 -> "TAG_LAST_KEYCODE"</div>
<div>
<br /> <br />
<span style="color: red;">#Android : aapt 使用</span><br />
1. aapt l[ist] [-v] [-a] file.{zip,jar,apk}<br />List contents of Zip-compatible archive.<br />1.1 列出壓縮檔目錄<br />aapt l <file_path.apk><br />參數:<br />-v:會以table的形式輸出目錄,table的表目有:Length、Method、Size、Ratio、Date、Time、CRC-32、Name。<br />其中Method表示壓縮形式,有:Deflate及Stored兩種,即該Zip目錄採用的演算法是壓縮模式還是存儲模式;可以看出resources.arsc、*.png採用壓縮模式,而其它採用壓縮模式。<br />Ratio表示壓縮率。CRC-32未明其意,Sodino盼指教。<br /><br />-a:會詳細輸出所有目錄的內容。<br /><br />2. aapt d[ump] [--values] WHAT file.{apk} [asset [asset ...]]<br />badging Print the label and icon for the app declared in APK.<br />permissions Print the permissions from the APK.<br />resources Print the resource table from the APK.<br />configurations Print the configurations in the APK.<br />xmltree Print the compiled xmls in the given assets.<br />xmlstrings Print the strings of the given compiled xml assets.<br /><br />2.1 查看apk包的packageName、versionCode、applicationLabel、launcherActivity、permission等各種詳細資訊<br />aapt dump badging <file_path.apk><br /><br />2.2 查看許可權<br />aapt dump permissions <file_path.apk><br /><br />2.3 查看資源清單<br />aapt dump resources <file_path.apk><br />一般都會輸出很多的資訊,如要全部查看,請用下麵這兩句:<br />aapt dump resources <file_path.apk> > sodino.txt<br />sodino.txt<br />這樣會把所有的資訊通過重定向符">"輸出到sodino.txt檔中,然後再打開該檔即可查看。<br /><br />2.4 查看apk配置資訊<br />aapt dump configurations <file_path.apk><br /><br /><br />2.5 查看指定apk的指定xml檔。<br />aapt dump xmltree <file_path.apk> res/***.xml<br />以樹形結構輸出的xml資訊。<br />aapt dump xmlstrings <file_path.apk> res/***.xml<br />輸出xml檔中所有的字串資訊。<br /><br /><br /><br /><br />Reference:<br />1. <a href="http://fecbob.pixnet.net/blog/post/35827899-android-aapt%E4%BD%BF%E7%94%A8">http://fecbob.pixnet.net/blog/post/35827899-android-aapt%E4%BD%BF%E7%94%A8</a><b style="color: red; font-size: x-small;"><br /></b></div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0tag:blogger.com,1999:blog-7057734095102577753.post-52326330036572064292013-05-15T22:57:00.000-07:002013-05-16T18:39:45.146-07:00SELinux Notes -- Part I, Overview<table border="0" cellpadding="0" cellspacing="0" style="margin-left: 15px;"><tbody>
<tr>
<td class="v2" colspan="3" height="20"><b>SELinux by Example: Using Security Enhanced
Linux</b></td></tr>
<tr>
<td class="v2" colspan="3" height="18">By <a class="v1" href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" target="_new">Frank Mayer,</a>, <a class="v1" href="https://www.blogger.com/blogger.g?blogID=7057734095102577753" target="_new">Karl MacMillan,</a>, David Caplan<br />
<br />
=======================================================<br />
<br />
<ul>
<li>
<div class="docList">
SELinux access control is based on a security context
associated with all system resources including processes. The security context
contains three elements: user, role, and type identifiers. The type identifier
is the primary basis for access control.</div>
<div class="docList">
In SELinux, type enforcement is the primary access control
feature. Access is granted between subjects (that is, processes) and objects by
specifying <tt>allow</tt> rules that have the subject's type (also called a
domain type) as the source and the object's type as the target. Access is
granted for specified object classes using a fine-grained set of permissions
defined for each object class.</div>
<div class="docList">
One of the key benefits of type enforcement is the ability to
control which programs may run with a given domain type, thereby allowing access
control down to individual programs (rather than the less-secure level of a
user). The capability for a program to enter into a domain (that is, run with a
given process type) is called domain transition and is tightly controlled by
SELinux <tt>allow</tt> rules. SELinux also allows domain transitions to occur
automatically through the <tt>type_transition</tt> rule.</div>
<div class="docList">
<br /></div>
</li>
<li>
<div class="docList">
SELinux does not directly use the role identifiers in a
security context for access control. Instead, all access is controlled based on
types. Roles are used to associate the allowed domain types into which a process
running on behalf of a user may transition. This allows sets of type enforcement
allowed capabilities to be grouped together and authorized for a user as a role.<br />
<br /></div>
<div class="docList">
</div>
</li>
</ul>
<h3 class="docSection1Title" id="title-IDAYHTNO">
Type Enforcement Access Control</h3>
<div>
<br /></div>
<div>
In SELinux, all access must be explicitly granted. SELinux allows <span class="docEmphasis">no access by default</span>, regardless of the Linux
user/group IDs. Yes, this means that there is no default superuser in SELinux,
unlike root in standard Linux.</div>
<div>
<br /></div>
<div>
<div class="docText">
An <tt>allow</tt> rule has four elements:</div>
<ul>
<li>
<div class="docText">
<span class="docEmphasis">Source type(s)</span> Usually the
domain type of a process attempting access</div>
</li>
<li>
<div class="docText">
<span class="docEmphasis">Target type(s)</span> The type of an
object being accessed by the process</div>
</li>
<li>
<div class="docText">
<span class="docEmphasis">Object class(es)</span> The class of
object that the specified access is permitted</div>
</li>
<li>
<div class="docText">
<span class="docEmphasis">Permission(s)</span> The kind of access
that the source type is allowed to the target type for the indicated object
classes</div>
</li>
</ul>
<div class="docText">
</div>
<div class="docText">
As an example, take the following rule:</div>
<div class="docText">
<pre>allow user_t bin_t : file {read execute getattr};</pre>
<pre></pre>
</div>
<br />
<div class="docText">
<span style="font-family: inherit;">The translation of this rule would be as follows:</span><br />
<br />
<div class="docText">
A process with a domain type of <tt>user_t</tt> can read,
execute, or get attributes for a file object with a type of <tt>bin_t</tt>.</div>
<br />
<span style="font-family: inherit;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1TIPADrmOGDfU5STuycWm7PoGMdaM6lW4XovsqVxJSRJ9iUf3rSn5e4q3l1THtDwmIccNMX6DNTwE7RINVtts8ZcJusTHNKdTDoCOTOyUfNqivUeHNrfTXd3mx0BjyRhRGi460O4mW37R/s1600/allow+rule.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1TIPADrmOGDfU5STuycWm7PoGMdaM6lW4XovsqVxJSRJ9iUf3rSn5e4q3l1THtDwmIccNMX6DNTwE7RINVtts8ZcJusTHNKdTDoCOTOyUfNqivUeHNrfTXd3mx0BjyRhRGi460O4mW37R/s1600/allow+rule.jpg" width="320" /></a></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Example: passwd program</span><br />
<span style="color: blue; font-family: inherit;">allow passwd_t shadow_t : file {ioctl read write create getattr setattr lock relabelfrom relabelto append unlink link rename};</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTpnMUbzHfYoeYYNvU5a8hT9B3_GWMtP1WDFA_DZDev6ZJ_hl4BKrkuK7MJ6el5y8B4RY28DPFjAeskH50gcbuWMJR-b2_Ch2dX9DyJZdf6vepdtXjadMLZ_ERJ3MMtq4TJOFW9dIdCa9M/s1600/passwd.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="165" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTpnMUbzHfYoeYYNvU5a8hT9B3_GWMtP1WDFA_DZDev6ZJ_hl4BKrkuK7MJ6el5y8B4RY28DPFjAeskH50gcbuWMJR-b2_Ch2dX9DyJZdf6vepdtXjadMLZ_ERJ3MMtq4TJOFW9dIdCa9M/s1600/passwd.jpg" width="320" /></a></div>
<span style="font-family: inherit;"><br /></span>
<br />
<div class="docText">
<a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1051"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1052"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1251"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2030"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2031"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2078"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2079"></a>In this example, we defined two types.
The <tt>passwd_t</tt> type is a domain type intended for use by the password
program. The <tt>shadow_t</tt> type is the type for the shadow password file. If
we examine such a file on disk, we would see something like this:</div>
<div class="docText">
<pre># <span class="docEmphStrong">ls -Z /etc/shadow</span>
-r---- root root system_u:object_r:shadow_t shadow</pre>
</div>
<br />
<div class="docText">
Likewise, examining a process running the password program
under this policy would yield this:</div>
<div class="docText">
<pre># <span class="docEmphStrong">ps -aZ</span>
joe:user_r:passwd_t 16532 pts/0 00:00:00 passwd</pre>
</div>
<br />
<div class="docText">
The purpose of this rule is to give the <tt>passwd</tt> process' domain type
(<tt>passwd_t</tt>) the access to the shadow's file type (<tt>shadow_t</tt>)
needed to allow the process to move and create a new shadow password file.</div>
<div class="docText">
<br /></div>
<br />
<span style="font-family: inherit;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimIJcLstXx4pPoYKCryLvdzRprKRMGfN1VUtToYoExax1Ay9fNT_IKUVZbgpssR0mqChXpX1-K4Ycr8sgaYByCnrfO5IJDVWldqJiwVrGK1Nulh-57BBiVn-ZFNSF2_MQ_pUopeRz-DtGQ/s1600/psswd-selinux.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimIJcLstXx4pPoYKCryLvdzRprKRMGfN1VUtToYoExax1Ay9fNT_IKUVZbgpssR0mqChXpX1-K4Ycr8sgaYByCnrfO5IJDVWldqJiwVrGK1Nulh-57BBiVn-ZFNSF2_MQ_pUopeRz-DtGQ/s1600/psswd-selinux.png" width="400" /></a></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Above figure shows an example of passwd program security in SELinux. </span>The first rule is as follows:<br />
<span style="color: blue;">allow user_t passwd_exec_t : file {getattr execute};</span><br />
<br />
What this rule does is allow Joe's shell (user_t) to initiate an execve() system call on the passwd executable file (passwd_exec_t). The SELinux execute file permission is essentially the same permission as x access for files in standard Linux.<br />
<br />
<br />
The next allow rules is<br />
<span style="color: blue;">allow passwd_t passwd_exec_t : file entrypoint;</span><br />
<br />
This rule provides entrypoint access to the passwd_t domain. The entrypoint permission is a rather valuable permission in SELinux. What this permission does is define which executable files (and therefore which programs) may "enter" a domain.<br />
<br />
<br />
<br />
Let's now look at the final rule:<br />
<span style="color: blue;">allow user_t passwd_t : process transition;</span><br />
<br />
This is the first allow rule we have seen that did not provide access to file objects. In this case, the object class is process, meaning the object class representing processes. Recall that all system resources are encapsulated in an object class.<br />
<br />
<br />
<br />
<br />
These three rules together provide the necessary access for a domain transition to occur. For a domain transition to succeed, all three rules are necessary; alone, none is sufficient. Therefore, a domain transition is allowed only when the following three conditions are true:<br />
<br />
<ol>
<li>The process' new domain type has enTRypoint access to an executable file type.</li>
<li>The process' current (or old) domain type has execute access to the entry point file type.</li>
<li>The process' current domain type has transition access to the new domain type.</li>
</ol>
<br />
When all three of these permissions are permitted in a TE policy, a domain transition may occur. Further, with the use of the entrypoint permission on executable files, we have the power to strictly control which programs can run with a given domain type. The execve() system call is the only way to change a domain type, giving the policy writer great control over an individual program's access to privilege, regardless of the user who may be invoking the program.</div>
</div>
</td></tr>
</tbody></table>
<div>
<br />
<br />
<h3>
Default Domain Transitions: type_transition Statement</h3>
<br />
<br />
To support domain transitions occurring by default (as we want in the case of the password program), we need to introduce a new rule, the type transition rule (type_transition). This rule provides a means for the SELinux policy to specify default transitions that should be attempted if an explicit transition was not requested.<br />
<br />
<span style="color: blue;"><i>type_transition user_t passwd_exec_t : process passwd_t;</i></span><br />
<br />
The syntax of this rule differs from the allow rule. There are still source and target types (user_t and passwd_exec_t, respectively) and an object class (process). However, instead of permissions, we have a third type, the default type (passwd_t).<br />
Type_transition rules are used for multiple different purposes relating to default type changes.<br />
<br />
The type_transition rule indicates that, by default on an execve() system call, if the calling process' domain type is user_t and the executable file's type is passwd_exec_t , a domain transition to a new domain type (passwd_t) will be attempted.<br />
<br />
<br /></div>
<div>
The type_transition rule allows the policy writer to cause default domain transitions to be initiated without explicit user input. This makes type enforcement less obtrusive to the user.<br />
<br />
<br />
<h3>
Role-Based Access Control (RBAC)</h3>
<div>
<div>
<br /></div>
<div>
The RBAC feature of SELinux is built upon type enforcement; access control in SELinux is primarily via type enforcement. Roles limit the types to which a process may transition based on the role identifier in the process' security context.</div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidRTti0xUlWGULMuxPUeYKURUMYLaFoEI1kJ5D4sYublN4wV-CKAC097FA38TG4oMBRKNthxed7n8YhyphenhyphenvNAu0lUyUAj4QS41VPX9o_4gyE0fUoQmjs656RQjkR_JTu9Pju2jRfPCOT2yFJ/s1600/roles+in+domain.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="205" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidRTti0xUlWGULMuxPUeYKURUMYLaFoEI1kJ5D4sYublN4wV-CKAC097FA38TG4oMBRKNthxed7n8YhyphenhyphenvNAu0lUyUAj4QS41VPX9o_4gyE0fUoQmjs656RQjkR_JTu9Pju2jRfPCOT2yFJ/s1600/roles+in+domain.png" width="320" /></a></div>
<div>
<div class="docText">
<br /></div>
<div class="docText">
We have added the role portion (<tt>user_r</tt>) of the
security contexts for the processes depicted. We also added a new rule,
specifically the <span class="docEmphasis">role statement</span>:</div>
<div class="docText">
<pre>role user_r type passwd_t;</pre>
</div>
<br />
<div class="docText">
The <tt>role</tt> statement declares role identifiers and
associates types with the declared role. The previous statement declares the
role <tt>user_r</tt> (if it has not already been declared in the policy) and
associates the type <tt>passwd_t</tt> with the role. What this association means
is that the <tt>passwd_t</tt> type is allowed to coexist in a security context
with the role <tt>user_r</tt>. Without this <tt>role</tt> statement, the new
context <tt>joe:user_r:passwd_t</tt> could not be created, and the
<tt>execve()</tt> system call would fail, even though the TE policy allows Joe's
type (<tt>user_t</tt>) all the necessary access.<br />
<br />
<span style="background-color: white;"><span style="font-family: inherit;">Role-based access control (RBAC) is a general security model that simplifies administration by assigning roles to users and then assigning permissions to those roles. RBAC in Security-Enhanced Linux (SELinux) acts as a layer of abstraction between the user and the underlying type-enforcement (TE) model, which provides highly granular access control but is not geared for ease of management.</span></span><br />
<span style="background-color: white;"><span style="font-family: inherit;"><br /></span></span>
<span style="background-color: white;"><span style="font-family: inherit;"><br /></span></span></div>
</div>
<div>
<br /></div>
<div>
<h3>
Multilevel Security in SELinux (MLS)</h3>
</div>
<div>
<div class="docText">
The <span class="docEmphasis">security level</span> used by MLS systems is a
combination of a hierarchical <span class="docEmphasis">sensitivity</span> and a
set (including the null set) of nonhierarchical <span class="docEmphasis">categories</span>. These sensitivities and categories are used
to reflect real information confidentiality or user clearances. In most SELinux
policies, the sensitivities (<tt>s0</tt>, <tt>s1</tt>, ...) and categories
(<tt>c0</tt>, <tt>c1</tt>, ...) are given generic names, leaving it to userspace
programs and libraries to assign user-meaningful names. (For example,
<tt>s0</tt> might be associated with UNCLASSIFIED and <tt>s1</tt> with
SECRET.)</div>
<div class="docText">
To support MLS, the security context is extended to include
security levels as such these:</div>
<div class="docText">
<pre>user:role:type:sensitivity[:category,...][-sensitivity[:category,...]]</pre>
</div>
<br />
<div class="docText">
Notice that the MLS security context must have at least one
security level (which is composed of a single sensitivity and zero or more
categories), but can include two security levels. These two security levels are
called <span class="docEmphasis">low</span> (or <span class="docEmphasis">current</span> for processes) and <span class="docEmphasis">high</span> (or <span class="docEmphasis">clearance</span> for
processes), respectively. If the high security level is missing, it is
considered to be the same value as the low (the most common situation).</div>
<div class="docText">
<br /></div>
<div class="docText">
There are four dominance operators that can relate two MLS security levels are as
follows:</div>
<table cellpadding="5" cellspacing="0" frame="void" rules="none">
<colgroup align="left" span="2">
<col width="150"></col>
<col width="350"></col></colgroup>
<thead>
<tr>
<th align="left" class="thead" scope="col" valign="top"><div class="docText">
</div>
</th>
<th align="left" class="thead" scope="col" valign="top"><div class="docText">
</div>
</th></tr>
</thead>
<tbody>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis">dom:</span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
(<span class="docEmphasis">dominates</span>) SL1 <span class="docEmphasis">dom</span> SL2 if the sensitivity of SL1 is <span class="docEmphasis">higher or equal to</span> the sensitivity of SL2, <span class="docEmphasis">and</span> the categories of SL1 are a <span class="docEmphasis">superset</span> of the categories of SL2.</div>
</td></tr>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis">domby:</span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
(<span class="docEmphasis">dominated by</span>) SL1 <span class="docEmphasis">domby</span> SL2 if the sensitivity of SL1 is <span class="docEmphasis">lower than or equal to</span> the sensitivity of SL2, <span class="docEmphasis">and</span> the categories of SL1 are a <span class="docEmphasis">subset</span> of the categories of SL2.</div>
</td></tr>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis">eq:</span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
(<span class="docEmphasis">equals</span>) SL1 <span class="docEmphasis">eq</span> SL2 if the sensitivity of SL1 and SL2 are <span class="docEmphasis">equal, and</span> the categories of SL1 and SL2 are the <span class="docEmphasis">same set</span>.</div>
</td></tr>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis">incomp:</span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
(<span class="docEmphasis">incomparable</span> or <span class="docEmphasis">noncomparable</span>) SL1 <span class="docEmphasis">incomp</span> SL2 if the categories of SL1 and SL2 cannot be
compared (that is, neither is a subset of the
other).</div>
</td></tr>
</tbody></table>
</div>
<div>
<br />
<h3>
SELinux Policy Server Architecture</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_FxeBRSLdKvz9Xdrck7npWElMhE8oYzi68TsN2TXLcZTbH4e6UJ7k7H8kcgsXWk7WTBWjY3s5YhbDl_N7T9mSkt9RDAj6e_f0EztjEBvc0IdgOxhrnO6upYTzqjfjZviXc6e8YJBCb24z/s1600/SELinux+policy+server.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_FxeBRSLdKvz9Xdrck7npWElMhE8oYzi68TsN2TXLcZTbH4e6UJ7k7H8kcgsXWk7WTBWjY3s5YhbDl_N7T9mSkt9RDAj6e_f0EztjEBvc0IdgOxhrnO6upYTzqjfjZviXc6e8YJBCb24z/s640/SELinux+policy+server.png" width="640" /></a></div>
<br />
In the policy server architecture, all manipulation and management of the
overall <span class="docEmphasis">system policy</span> is controlled through the
<span class="docEmphasis">policy management server</span> (PMS). The PMS is itself
a userspace object manager in that it creates object classes representing policy
resources and enforces a fine-grained access control policy over those
resources.<br />
<br />
With the PMS, you can now allow access to portions of the policy and limit
access to others. For example, the SELinux policy can allow user management
tools to add users and make role assignments, but not change type enforcement
<tt>allow</tt> rules. Better yet, you can authorize a database server to change
<span class="docEmphasis">type enforcement</span> (TE) rules relating to its
object classes and types, but not those of the kernel. Internally, the PMS is
designed to use another recent new feature of SELinux, <span class="docEmphasis">loadable policy modules.</span><br />
<br />
<br />
<div class="docText">
The second major function of the PMS is to split the system
policy into kernel and user portions and load them respectively into the kernel
security server and <a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1094"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1454"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle1708"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2160"></a><a href="http://www.blogger.com/blogger.g?blogID=7057734095102577753" name="iddle2161"></a><span class="docEmphasis">userspace security server</span> (USSS). In this way, the
kernel is not made aware of rules and object classes of concern only to
userspace object managers. Userspace object managers query the USSS and not the
kernel. AVCs in various userspace object managers register with the USSS (and
not the kernel) for policy update and cache coherency functions.</div>
<div class="docText">
<br /></div>
<br />
<br />
<h3 class="docSection1Title" id="title-IDAVUYGB">
SELinux Policy Language</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjomgxGeI5PQUyxbiBGwdsTjcV3vZNNAdl2r_ebL22arSo5P0zRCRMsqXUMqfw4xgqghiGeIYHe_DS8_yHn6n8y1HEUEOzkq5iU3XcTQGGjL8cW7Q6IevrCkN84bPVFru0G_oSiQDwjUPP7/s1600/policy.conf.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjomgxGeI5PQUyxbiBGwdsTjcV3vZNNAdl2r_ebL22arSo5P0zRCRMsqXUMqfw4xgqghiGeIYHe_DS8_yHn6n8y1HEUEOzkq5iU3XcTQGGjL8cW7Q6IevrCkN84bPVFru0G_oSiQDwjUPP7/s400/policy.conf.png" width="196" /></a></div>
<div>
<br /></div>
<div>
The first section of a policy source file defines the object classes to the
security server. This section also defines the permissions for each object
class. For the kernel, these classes are directly related to kernel source
files. <span style="background-color: yellow;">In general, as an SELinux policy writer you would never change or modify
the object class and permission definitions. </span></div>
<div>
<br /></div>
<div>
The next section contains the type enforcement statements, which is by far the
largest portion of an SELinux policy. This is the section that policy writers
spend most of their time writing. It contains all the type declarations and all
the TE rules (including all <tt>allow</tt>, <tt>type_transition</tt>, and other
TE rules).</div>
<div>
<br /></div>
<div>
The next section of a policy source file contains the constraints. Constraints
provide a means of further limiting the TE policy beyond what the TE rules
permit. The <span class="docEmphasis">multilevel security</span> (MLS) policy, for
example, is implemented as a set of constraints.</div>
<div>
<br /></div>
<div>
The last section of a policy file contains labeling specifications. All objects
must be labeled with a security context for SELinux to enforce access control.
This section tells SELinux how to treat filesystems for the purpose of labeling
and contains the rules for labeling transient objects that are created at
runtime. A separate related mechanism, called a <span class="docEmphasis">file
contexts file</span>, is used to initialize the security context labeling of
files, directories, and other objects on permanent filesystems.</div>
<div>
<br /></div>
<div>
<h3>
Building and Installing Monolithic Policies</h3>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5QHtoYwKXBVEwvSg8odyXmPZ7B5QjvaMjhZPNogZikvVoGJjgEaopHQU90gJ2EKXyS-Qv0We03W3Tn4uD6VOsxAZnmzxYB0ICFvpsdiRkPnsxaPi50dHpJHJAtrx5XELY4TrrCmOdfB6Y/s1600/build+and+load+policy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="489" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5QHtoYwKXBVEwvSg8odyXmPZ7B5QjvaMjhZPNogZikvVoGJjgEaopHQU90gJ2EKXyS-Qv0We03W3Tn4uD6VOsxAZnmzxYB0ICFvpsdiRkPnsxaPi50dHpJHJAtrx5XELY4TrrCmOdfB6Y/s640/build+and+load+policy.png" width="640" /></a></div>
<div>
<br /></div>
<div>
Above figure shows a typical way that a policy is constructed.</div>
<div>
<br /></div>
<div>
Starting from the left side of this figure, you have the source files for the
policy broken down into many tens of individual source modules. Compile the source policy using <tt>checkpolicy</tt> into a binary policy file, <tt>load_policy</tt> program is then used to load the binary policy file into
the kernel, which then enforces access control based on the policy rules.</div>
<div>
<table cellpadding="5" cellspacing="0" frame="void" rules="none"><tbody>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis"><tt>policy</tt></span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
Make <tt>policy.conf</tt> and <tt>policy.[ver]</tt> locally to
test the compilation and check for error.</div>
</td></tr>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis"><tt>install</tt></span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
Do everything that <tt>make policy</tt> does plus install the
binary policy file such that it will be loaded into the kernel at boot time and
the policy configuration files.</div>
</td></tr>
<tr>
<td align="left" class="docTableCell" valign="top"><div class="docText">
<span class="docEmphasis"><tt>load</tt></span></div>
</td>
<td align="left" class="docTableCell" valign="top"><div class="docText">
Do everything that <tt>make policy</tt> does plus immediately
load the binary policy file into the kernel as the active access control policy
and install the <tt>file_contexts</tt>
file.</div>
</td></tr>
</tbody></table>
<br /></div>
<div>
<br /></div>
<div>
<a name='more'></a><ul>
<li>
<div class="docList">
SELinux is implemented as an LSM module in the kernel. SELinux
uses LSM hooks throughout the kernel to control access to kernel resources.
Access decisions are made by the SELinux security server, which is part of the
SELinux LSM module. The security policy enforced by the security server is
loaded into the kernel via a privileged userspace interface. The AVC provides
performance improvement for access validation.</div>
<div class="docList">
The SELinux framework also supports userspace object managers
through the <tt>libselinux</tt> library. In its basic form, the kernel security
server directly provides access validation, whereas the library contains a
per-process AVC. This approach requires the kernel to hold the policy for all
userspace managers and to be aware of all userspace object classes.</div>
</li>
<li>
<div class="docList">
The emerging policy server architecture enhances support for
userspace object managers by providing a userspace security server that will
enforce all portions of the policy relating to userspace objects, thereby
relieving the kernel of its need to know of userspace object classes and policy
rules. The policy server will also provide fine-grained access control to the
policy itself, allowing greater distribution of policy management
authority.</div>
</li>
</ul>
</div>
</div>
</div>
Morrishttp://www.blogger.com/profile/08528778345368339739noreply@blogger.com0