Jekyll2024-01-30T00:02:26-08:00https://prinsphield.github.io/feed.xmlTaihong XiaoSoftware Engineer at Google ResearchTaihong Xiaotaihong.xiao@gmail.comAI玩微信跳一跳的正确姿势2018-01-04T00:00:00-08:002018-01-04T00:00:00-08:00https://prinsphield.github.io/posts/2018/01/wechat_jump<p>最近,微信小游戏跳一跳可以说是火遍了全国,从小孩子到大孩子仿佛每一个人都在刷跳一跳,作为无(zhi)所(hui)不(ban)能(zhuan)的AI程序员,我们在想,能不能用人工智能(AI)和计算机视觉(CV)的方法来玩一玩这个游戏?于是,我们开发了微信跳一跳Auto-Jump算法,重新定义了玩跳一跳的正确姿势,我们的算法不仅远远超越了人类的水平,在速度和准确度上也远远超越了目前已知的所有算法,可以说是跳一跳界的state-of-the-art,下面我们详细介绍我们的算法。</p>
<p>算法的第一步是获取手机屏幕的截图并可以控制手机的触控操作,我们的<a href="https://github.com/Prinsphield/Wechat_AutoJump">github</a>仓库里详细介绍了针对Android和IOS手机的配置方法,你只需要按照将手机连接电脑,按照教程执行就可以完成配置。在获取到屏幕截图之后,就是个简单的视觉问题。我们需要找的就是小人的位置和下一次需要跳的台面的中心。
如图所示,绿色的点代表小人当前的位置,红点代表目标位置。</p>
<div align="center">
<img align="center" src="/extra/wechat_jump/state_000.png" width="250" alt="state_000" />
<img align="center" src="/extra/wechat_jump/state_000_res.png" width="250" alt="state_000_res" />
</div>
<p><br /></p>
<h3 id="多尺度搜索multiscale-search">多尺度搜索(Multiscale Search)</h3>
<p>这个问题可以有非常多的方法去解,为了糙快猛地刷上榜,我一开始用的方式是多尺度搜索。
我随便找了一张图,把小人抠出来,就像下面这样。</p>
<div align="center">
<img align="center" src="https://raw.githubusercontent.com/Prinsphield/Wechat_AutoJump/master/resource/player.png" width="50" alt="player" />
</div>
<p><br /></p>
<p>另外,我注意到小人在屏幕的不同位置,大小略有不同,所以我设计了多尺度的搜索,用不同大小的进行匹配,
最后选取置信度(confidence score)最高的。</p>
<p>多尺度搜索的代码长这样</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">multi_scale_search</span><span class="p">(</span><span class="n">pivot</span><span class="p">,</span> <span class="n">screen</span><span class="p">,</span> <span class="nb">range</span><span class="o">=</span><span class="mf">0.3</span><span class="p">,</span> <span class="n">num</span><span class="o">=</span><span class="mi">10</span><span class="p">):</span>
<span class="n">H</span><span class="p">,</span> <span class="n">W</span> <span class="o">=</span> <span class="n">screen</span><span class="p">.</span><span class="n">shape</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span>
<span class="n">h</span><span class="p">,</span> <span class="n">w</span> <span class="o">=</span> <span class="n">pivot</span><span class="p">.</span><span class="n">shape</span><span class="p">[:</span><span class="mi">2</span><span class="p">]</span>
<span class="n">found</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">for</span> <span class="n">scale</span> <span class="ow">in</span> <span class="n">np</span><span class="p">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="nb">range</span><span class="p">,</span> <span class="mi">1</span><span class="o">+</span><span class="nb">range</span><span class="p">,</span> <span class="n">num</span><span class="p">)[::</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
<span class="n">resized</span> <span class="o">=</span> <span class="n">cv2</span><span class="p">.</span><span class="n">resize</span><span class="p">(</span><span class="n">screen</span><span class="p">,</span> <span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">W</span> <span class="o">*</span> <span class="n">scale</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">H</span> <span class="o">*</span> <span class="n">scale</span><span class="p">)))</span>
<span class="n">r</span> <span class="o">=</span> <span class="n">W</span> <span class="o">/</span> <span class="nb">float</span><span class="p">(</span><span class="n">resized</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="k">if</span> <span class="n">resized</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o"><</span> <span class="n">h</span> <span class="ow">or</span> <span class="n">resized</span><span class="p">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o"><</span> <span class="n">w</span><span class="p">:</span>
<span class="k">break</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">cv2</span><span class="p">.</span><span class="n">matchTemplate</span><span class="p">(</span><span class="n">resized</span><span class="p">,</span> <span class="n">pivot</span><span class="p">,</span> <span class="n">cv2</span><span class="p">.</span><span class="n">TM_CCOEFF_NORMED</span><span class="p">)</span>
<span class="n">loc</span> <span class="o">=</span> <span class="n">np</span><span class="p">.</span><span class="n">where</span><span class="p">(</span><span class="n">res</span> <span class="o">>=</span> <span class="n">res</span><span class="p">.</span><span class="nb">max</span><span class="p">())</span>
<span class="n">pos_h</span><span class="p">,</span> <span class="n">pos_w</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="o">*</span><span class="n">loc</span><span class="p">))[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">if</span> <span class="n">found</span> <span class="ow">is</span> <span class="bp">None</span> <span class="ow">or</span> <span class="n">res</span><span class="p">.</span><span class="nb">max</span><span class="p">()</span> <span class="o">></span> <span class="n">found</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
<span class="n">found</span> <span class="o">=</span> <span class="p">(</span><span class="n">pos_h</span><span class="p">,</span> <span class="n">pos_w</span><span class="p">,</span> <span class="n">r</span><span class="p">,</span> <span class="n">res</span><span class="p">.</span><span class="nb">max</span><span class="p">())</span>
<span class="k">if</span> <span class="n">found</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span> <span class="k">return</span> <span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">,</span><span class="mi">0</span><span class="p">)</span>
<span class="n">pos_h</span><span class="p">,</span> <span class="n">pos_w</span><span class="p">,</span> <span class="n">r</span><span class="p">,</span> <span class="n">score</span> <span class="o">=</span> <span class="n">found</span>
<span class="n">start_h</span><span class="p">,</span> <span class="n">start_w</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">pos_h</span> <span class="o">*</span> <span class="n">r</span><span class="p">),</span> <span class="nb">int</span><span class="p">(</span><span class="n">pos_w</span> <span class="o">*</span> <span class="n">r</span><span class="p">)</span>
<span class="n">end_h</span><span class="p">,</span> <span class="n">end_w</span> <span class="o">=</span> <span class="nb">int</span><span class="p">((</span><span class="n">pos_h</span> <span class="o">+</span> <span class="n">h</span><span class="p">)</span> <span class="o">*</span> <span class="n">r</span><span class="p">),</span> <span class="nb">int</span><span class="p">((</span><span class="n">pos_w</span> <span class="o">+</span> <span class="n">w</span><span class="p">)</span> <span class="o">*</span> <span class="n">r</span><span class="p">)</span>
<span class="k">return</span> <span class="p">[</span><span class="n">start_h</span><span class="p">,</span> <span class="n">start_w</span><span class="p">,</span> <span class="n">end_h</span><span class="p">,</span> <span class="n">end_w</span><span class="p">,</span> <span class="n">score</span><span class="p">]</span>
</code></pre></div></div>
<p>我们来试一试,效果还不错,应该说是又快又好,我所有的实验中找小人从来没有失误。
不过这里的位置框的底部中心并不是小人的位置,真实的位置是在那之上一些。</p>
<div align="center">
<img align="center" src="/extra/wechat_jump/player_detection.png" width="250" alt="player" />
</div>
<p><br /></p>
<p>同理,目标台面也可以用这种办法搜索,但是我们需要收集一些不同的台面,有圆形的,方形的,
便利店,井盖,棱柱等等。由于数量一多,加上多尺度的原因,速度上会慢下来。这时候,我们就需要想办法加速了。
首先可以注意到目标位置始终在小人的位置的上面,所以可以操作的一点就是在找到小人位置之后把小人位置以下的
部分都舍弃掉,这样可以减少搜索空间。</p>
<p>但是这还是不够,我们需要进一步去挖掘游戏里的故事。小人和目标台面基本上是关于屏幕中心对称的位置的。
这提供了一个非常好的思路去缩小搜索空间。假设屏幕分辨率是(1280,720)的,小人底部的位置是(h1, w1),
那么关于中心对称点的位置就是(1280 - h1, 720 - w1),以这个点为中心的一个边长300的正方形内,我们再去多尺度搜索目标位置,
就会又快有准了。效果见下图,蓝色框是(300,300)的搜索区域,红色框是搜到的台面,矩形中心就是目标点的坐标了。</p>
<div align="center">
<img align="center" src="/extra/wechat_jump/target_detection.png" width="250" alt="player" />
</div>
<p><br /></p>
<h3 id="加速的奇技淫巧fast-search">加速的奇技淫巧(Fast-Search)</h3>
<p>玩游戏需要细心观察。我们可以发现,小人上一次如果跳到台面中心,那么下一次目标台面的中心会有一个白点,就像刚才
所展示的图里的。更加细心的人会发现,白点的RGB值是(245,245,245),这就让我找到了一个非常简单并且
高效的方式,就是直接去搜索这个白点,注意到白点是一个连通区域,像素值为(245,245,245)的像素个数稳定在
280-310之间,所以我们可以利用这个去直接找到目标的位置。这种方式只在前一次跳到中心的时候可以用,不过没有关系,
我们每次都可以试一试这个不花时间的方法,不行再考虑多尺度搜索。</p>
<p>讲到这里,我们的方法已经可以运行的非常出色了,基本上是一个永动机。下面是用我的手机玩了一个半小时左右,跳了859次的状态,
我们的方法正确的计算出来了小人的位置和目标位置,不过我选择狗带了,因为手机卡的已经不行了。</p>
<div align="center">
<img align="center" src="https://raw.githubusercontent.com/Prinsphield/Wechat_AutoJump/master/resource/state_859.png" width="250" alt="state_859" />
<img align="center" src="https://raw.githubusercontent.com/Prinsphield/Wechat_AutoJump/master/resource/state_859_res.png" width="250" alt="state_859_res" />
<img align="center" src="https://raw.githubusercontent.com/Prinsphield/Wechat_AutoJump/master/resource/sota.png" width="250" alt="sota" />
</div>
<p><br /></p>
<p>这里有一个示例视频,欢迎观看!</p>
<!-- [![微信跳一跳](https://img.youtube.com/vi/OeTI2Kx8Ehc/0.jpg)](https://youtu.be/OeTI2Kx8Ehc "自动玩微信小游戏跳一跳") -->
<iframe width="560" height="315" src="https://www.youtube.com/embed/OeTI2Kx8Ehc" frameborder="0" gesture="media" allow="encrypted-media" allowfullscreen=""></iframe>
<p>到这里就结束了吗?那我们和业余玩家有什么区别?下面进入正经的学术时间,非战斗人员请迅速撤离!</p>
<h3 id="cnn-coarse-to-fine-模型">CNN Coarse-to-Fine 模型</h3>
<p>考虑到IOS设备由于屏幕抓取方案的限制(WebDriverAgent获得的截图经过了压缩,图像像素受损,不再是原来的像素值,原因不详,欢迎了解详情的小伙伴提出改进意见~)无法使用fast-search,同时为了兼容多分辨率设备,我们使用卷积神经网络构建了一个更快更鲁棒的目标检测模型,下面分数据采集与预处理,coarse模型,fine模型,cascade四部分介绍我们的算法。</p>
<h4 id="数据采集与预处理">数据采集与预处理</h4>
<p>基于我们非常准确的multiscale-search、fast-search模型,我们采集了7次实验数据,共计大约3000张屏幕截图,每一张截图均带有目标位置标注,对于每一张图,我们进行了两种不同的预处理方式,并分别用于训练coarse模型和fine模型,下面分别介绍两种不同的预处理方式。</p>
<h5 id="coarse-模型数据预处理">coarse 模型数据预处理</h5>
<p>由于每一张图像中真正对于当前判断有意义的区域只在屏幕中央位置,即人和目标物体所在的位置,因此,每一张截图的上下两部分都是没有意义的,因此,我们将采集到的大小为(1280, 720)的图像沿h方向上下各截去(320, 720)大小,只保留中心(640, 720)的图像作为训练数据。</p>
<p>我们观察到,游戏中,每一次当小人落在目标物中心位置时,下一个目标物的中心会出现一个白色的圆点,</p>
<div align="center">
<img align="center" src="/extra/wechat_jump/state_000.png" width="250" alt="state_000" />
</div>
<p><br />
考虑到训练数据中fast-search会产生大量有白点的数据,为了杜绝白色圆点对网络训练的干扰,我们对每一张图进行了去白点操作,具体做法是,用白点周围的纯色像素填充白点区域。</p>
<h5 id="fine-模型数据预处理">fine 模型数据预处理</h5>
<p>为了进一步提升模型的精度,我们为fine模型建立了数据集,对训练集中的每一张图,在目标点附近截取320*320大小的一块作为训练数据,</p>
<div align="center">
<img align="center" src="/extra/wechat_jump/piece.png" width="250" alt="piece" />
</div>
<p><br />
为了防止网络学到trivial的结果,我们对每一张图增加了50像素的随机偏移。fine模型数据同样进行了去白点操作。</p>
<h4 id="coarse-模型">coarse 模型</h4>
<p>我们把这一问题看成了回归问题,coarse模型使用一个卷积神经网络回归目标的位置,</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">img</span><span class="p">,</span> <span class="n">is_training</span><span class="p">,</span> <span class="n">keep_prob</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'coarse'</span><span class="p">):</span>
<span class="k">with</span> <span class="n">tf</span><span class="p">.</span><span class="n">name_scope</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">with</span> <span class="n">tf</span><span class="p">.</span><span class="n">variable_scope</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">conv2d</span><span class="p">(</span><span class="s">'conv1'</span><span class="p">,</span> <span class="n">img</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="bp">self</span><span class="p">.</span><span class="n">input_channle</span><span class="p">,</span> <span class="mi">16</span><span class="p">],</span> <span class="mi">2</span><span class="p">)</span>
<span class="c1"># out = tf.layers.batch_normalization(out, name='bn1', training=is_training)
</span> <span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">relu</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'relu1'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv2'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">32</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv3'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv4'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">7</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">64</span><span class="p">,</span> <span class="mi">128</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv5'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">9</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">128</span><span class="p">,</span> <span class="mi">256</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">256</span> <span class="o">*</span> <span class="mi">20</span> <span class="o">*</span> <span class="mi">23</span><span class="p">])</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_fc</span><span class="p">(</span><span class="s">'fc1'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">256</span> <span class="o">*</span> <span class="mi">20</span> <span class="o">*</span> <span class="mi">23</span><span class="p">,</span> <span class="mi">256</span><span class="p">],</span> <span class="n">keep_prob</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_fc</span><span class="p">(</span><span class="s">'fc2'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">256</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="n">keep_prob</span><span class="p">)</span>
<span class="k">return</span> <span class="n">out</span>
</code></pre></div></div>
<p>经过十小时的训练,coarse模型在测试集上达到了6像素的精度,实际测试精度大约为10像素,在测试机器(MacBook Pro Retina, 15-inch, Mid 2015, 2.2 GHz Intel Core i7)上inference时间0.4秒。这一模型可以很轻松的拿到超过1k的分数,这已经远远超过了人类水平和绝大多数自动算法的水平,日常娱乐完全够用,不过,你认为我们就此为止那就大错特错了~</p>
<h4 id="fine-模型">fine 模型</h4>
<p>fine模型结构与coarse模型类似,参数量稍大,fine模型作为对coarse模型的refine操作,</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">img</span><span class="p">,</span> <span class="n">is_training</span><span class="p">,</span> <span class="n">keep_prob</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'fine'</span><span class="p">):</span>
<span class="k">with</span> <span class="n">tf</span><span class="p">.</span><span class="n">name_scope</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">with</span> <span class="n">tf</span><span class="p">.</span><span class="n">variable_scope</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">conv2d</span><span class="p">(</span><span class="s">'conv1'</span><span class="p">,</span> <span class="n">img</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="bp">self</span><span class="p">.</span><span class="n">input_channle</span><span class="p">,</span> <span class="mi">16</span><span class="p">],</span> <span class="mi">2</span><span class="p">)</span>
<span class="c1"># out = tf.layers.batch_normalization(out, name='bn1', training=is_training)
</span> <span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">relu</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'relu1'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv2'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">64</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv3'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">64</span><span class="p">,</span> <span class="mi">128</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv4'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">7</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">128</span><span class="p">,</span> <span class="mi">256</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_conv_bn_relu</span><span class="p">(</span><span class="s">'conv5'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">9</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">256</span><span class="p">,</span> <span class="mi">512</span><span class="p">],</span> <span class="mi">1</span><span class="p">,</span> <span class="n">is_training</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">nn</span><span class="p">.</span><span class="n">max_pool</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">padding</span><span class="o">=</span><span class="s">'SAME'</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="n">tf</span><span class="p">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">512</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">10</span><span class="p">])</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_fc</span><span class="p">(</span><span class="s">'fc1'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">512</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">512</span><span class="p">],</span> <span class="n">keep_prob</span><span class="p">)</span>
<span class="n">out</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">make_fc</span><span class="p">(</span><span class="s">'fc2'</span><span class="p">,</span> <span class="n">out</span><span class="p">,</span> <span class="p">[</span><span class="mi">512</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="n">keep_prob</span><span class="p">)</span>
<span class="k">return</span> <span class="n">out</span>
</code></pre></div></div>
<p>经过十小时训练,fine模型测试集精度达到了0.5像素,实际测试精度大约为1像素,在测试机器上的inference时间0.2秒。</p>
<h4 id="cascade">cascade</h4>
<div align="center">
<img align="center" src="/extra/wechat_jump/we_jump_nn.png" width="750" alt="nn" />
</div>
<p><br /></p>
<div align="center">
<img align="center" src="/extra/wechat_jump/nn1.png" width="250" alt="nn" />
<img align="center" src="/extra/wechat_jump/nn2.png" width="250" alt="nn" />
<img align="center" src="/extra/wechat_jump/nn3.png" width="250" alt="nn" />
</div>
<p><br />
总体精度1像素左右,时间0.6秒。</p>
<h3 id="总结">总结</h3>
<p>针对这一问题,我们利用AI和CV技术,提出了合适适用于IOS和Android设备的完整解决方案,稍有技术背景的用户都可以实现成功配置、运行,我们提出了Multiscale-Search,Fast-Search,CNN Coarse-to-Fine三种解决这一问题的算法,三种算法相互配合,可以实现快速准确的搜索、跳跃,用户针对自己的设备稍加调整跳跃参数即可接近实现“永动机”。讲到这里,似乎可以宣布,我们的工作terminate了这个问题,微信小游戏跳一跳game over!</p>
<p><a href="https://github.com/Prinsphield/Wechat_AutoJump">代码链接</a></p>
<p><strong>友情提示:适度游戏益脑,沉迷游戏伤身,技术手段的乐趣在于技术本身而不在游戏排名,希望大家理性对待游戏排名和本文提出的技术,用游戏娱乐自己的生活</strong></p>
<p><strong>声明:本文提出的算法及开源代码符合MIT开源协议,以商业目的使用该算法造成的一切后果须由使用者本人承担</strong></p>
<h3 id="contributors">Contributors</h3>
<ul>
<li>Xiao Taihong <a href="mailto:xiaotaihong@126.com">xiaotaihong@126.com</a></li>
<li>An Jie <a href="mailto:jie.an@pku.edu.cn">jie.an@pku.edu.cn</a></li>
</ul>Taihong Xiaotaihong.xiao@gmail.com最近,微信小游戏跳一跳可以说是火遍了全国,从小孩子到大孩子仿佛每一个人都在刷跳一跳,作为无(zhi)所(hui)不(ban)能(zhuan)的AI程序员,我们在想,能不能用人工智能(AI)和计算机视觉(CV)的方法来玩一玩这个游戏?于是,我们开发了微信跳一跳Auto-Jump算法,重新定义了玩跳一跳的正确姿势,我们的算法不仅远远超越了人类的水平,在速度和准确度上也远远超越了目前已知的所有算法,可以说是跳一跳界的state-of-the-art,下面我们详细介绍我们的算法。A Mathematical View towards CycleGAN2017-07-05T00:00:00-07:002017-07-05T00:00:00-07:00https://prinsphield.github.io/posts/2017/07/math_view_cyclegan<p>Recently, CycleGAN is a very popular image translation method, which arouse many people’s interests. Lots of people are busy with reproducing it or designing interesting image applications by replacing the training data. However, few people have thought about its limitations, though the original <a href="https://arxiv.org/pdf/1703.10593.pdf">paper</a> gave some discussions about them.</p>
<p>A failure case of CycleGAN was given by the author himself.</p>
<div align="center">
<img src="https://junyanz.github.io/CycleGAN/images/failure_putin.jpg" width="450" alt="A failure case of CycleGAN." />
</div>
<div align="center">
A failure case of CycleGAN.
</div>
<h2 id="limitations-of-cyclegan">Limitations of CycleGAN</h2>
<p>We can actually obtain lots of information from this picture.</p>
<ul>
<li>CycleGAN is not able to disentangle the object from the background. Putin and background are both zebraized at the same time, as shown the in the picture.</li>
<li>CycleGAN is suitable for global image style transfer, but weak at doing object transfiguration.
What is object transfiguration? We want to segment a certain part from an image and seamlessly implant it into another images.
For example, I want to generate a smiling face of a person by imitating the smile from another smiling face.</li>
</ul>
<div align="center">
<img src="https://raw.githubusercontent.com/Prinsphield/GeneGAN/master/images/cross.jpg" width="450" alt="Smile Transfiguration" />
</div>
<div align="center">
Smile transfiguration.
</div>
<ul>
<li>Lack of diversity in generated images, or single-modal phenomenon. For example, we use CycleGAN to do image translation between two domains. One is facial images without eyeglasses, and the other on is facial images with eyeglasses. CycleGAN can generate novel images with eyeglasses from those images without eyeglasses, but the novel eyeglasses seems always to be a black sunglasses. This phenomenon was observed by Shuchang Zhou, who was very prescient. (see our paper <a href="https://arxiv.org/abs/1705.04932">GeneGAN</a>) I believe Jun-Yan Zhu, the author CycleGAN has also noticed this limitation. Otherwise, he would not publish another paper <a href="https://papers.nips.cc/paper/6650-toward-multimodal-image-to-image-translation.pdf">BiCycleGAN</a> to generate multi-modal images in NIPS 2017, right after ICCV 2017 submission deadline, though the idea of BiCycleGAN is somewhat simple and the training process is very similar to <a href="https://arxiv.org/abs/1611.06355">IcGAN</a>.</li>
<li>Weak at learning the shape of object. It is impossible if we want to use CycleGAN to generating a round object from a quadrate object.</li>
</ul>
<h2 id="the-reason-behind-it">The Reason Behind It</h2>
<p>What cause these limitations of CycleGAN? Recall the structure of CycleGAN, there are two image domains $X$ and $Y$.
The ultimate goal of CycleGAN is to learn two maps $G$ and $F$, where $G$ maps domain $X$ to domain $Y$, and $F$ maps
domain $Y$ to domain $X$.</p>
<div align="center">
<img src="/extra/cyclegan/cyclegan_framework.png" width="250" />
</div>
<div align="center">
CycleGAN framework.
</div>
<p>Recalling a simple result in topology, <a href="https://en.wikipedia.org/wiki/Invariance_of_domain">Invariance of domain</a> theorem states</p>
<blockquote>
<p>If $U$ is an open subset of $\mathbb{R}^n$ and $f:U\to \mathbb{R}^n$ is an injective continuous map, then $V = f(U)$ is an open
and $f$ is a homeomorphism between $U$ and $V$.</p>
</blockquote>
<p>We will leave out the proof since it uses tools of algebraic topology.
This theorem tells us an important consequence:</p>
<blockquote>
<p>$\mathbb{R}^n$ can not be homeomorphic to $\mathbb{R}^m$ if $m\neq n$.
Indeed, no non-empty open subset of $\mathbb{R}^n$ can be homeomorphic to any open subset of $\mathbb{R}^m$ if $m\neq n$.</p>
</blockquote>
<p>Back to our discussion of CycleGAN, $G$ and $F$ are generally neural networks with auto-encoder structures, therefore continuous.
(The composition of continuous map is continuous.)
The cycle consistency guarantees that $G$ and $F$ are inverse to each other.
Therefore the domain $X$ and domain $Y$ are homeomorphic.
According to the theorem of invariance of domain, the intrinsic dimensions of $X$ and $Y$ should be the same.</p>
<p><strong>This is the fundamental reason for its limitations!!!</strong>
Because the intrinsic dimensions of $X$ and $Y$ may not be the same.</p>
<p>For example, if we want to do image translation between domain $X$ of facial images with eyeglasses and domain $Y$ of facial images without eyeglasses. Let’s compare the intrinsic dimensions of two domains. The intrinsic dimension of domain $Y$ comes from the variety of facial images.
However, the intrinsic dimension of domain $X$ is more than that, because eyeglasses also varies, which increase the intrinsic dimensions.</p>
<p>In conclusion, the limitation of CycleGAN comes from the difference of intrinsic dimensions between source image domain and target image domain.
If you are interested, please read our paper <a href="https://arxiv.org/abs/1705.04932">GeneGAN</a> for an alternative method.</p>Taihong Xiaotaihong.xiao@gmail.comRecently, CycleGAN is a very popular image translation method, which arouse many people’s interests. Lots of people are busy with reproducing it or designing interesting image applications by replacing the training data. However, few people have thought about its limitations, though the original paper gave some discussions about them.An Overview on Optimization Algorithms in Deep Learning 12016-02-04T00:00:00-08:002016-02-04T00:00:00-08:00https://prinsphield.github.io/posts/2016/02/overview_opt_alg_deep_learning1<p>Recently, I have been learning about optimization algorithms in deep learning. And it is necessary, I think, to sum them up, so I plan to write a series of articles about different kinds of these algorithms. This article will mainly talk about the basic optimization algorithms used in machine learning and deep learning.</p>
<h2 id="gradient-descent">Gradient Descent</h2>
<p>Gradient descent is the most basic gradient-based algorithm to train a deep model. Once we get the function, let’s say $L$, that needs to optimize, especially when the function is convex, we can easily apply this method to approach the minimum of a function. This method involves updating the model parameters $\theta$ with a small step in the direction of the gradient of the objective function. For the case of supervised learning with data pairs $(x^{(i)}, y^{(i)})$, we have</p>
\[\theta\leftarrow \theta + \epsilon\nabla_\theta \sum_t L(f(x^{(i)};\theta), y^{(i)};\theta).\]
<p>where $\epsilon$ is the learning rate or the step size, that controls the size of the step the parameter takes in each iteration.</p>
<h2 id="stochastic-gradient-descent-sgd">Stochastic Gradient Descent (SGD)</h2>
<p>In spite of its impressive convergence property, batch gradient descent is rarely used in machine learning, because the cost of calculating the sum over gradient of each sample would be enormous when the number of training samples becomes large. A computationally efficient way is stochastic gradient descent in which we use the stochastic estimator of the gradient to perform its update. Based on the assumption that all samples are i.i.d, we sample one or a small suubset of $m$ training samples and compute their gradient. Then we use the gradient to update the parameter $\theta$.</p>
<p>When $m=1$, this algorithm is sometimes called <em>online gradient descent</em>. When $m>1$, the algorithm is sometimes called <em>minibatch SGD</em>. The algorithm is showed below.</p>
<p><img src="/extra/optimization/SGD.jpg" /></p>
<p>GD or SGD, cannot escape from the occasion: if the learning rate is too big, the weights may travel to and fro across the ravine and not converge to the minimum; if the learning rate is too small, it will take a long time to converge or coonverge to local minima. Thus, we need adjust the learning rate accordingly: if the error is falliing fairly consistently but slowly, increase the learning rate; if the error keeps getting worse or oscillates wildly, then we should reduce the learning rate. Is there any algorithms that adapt the learning rate automatically?</p>
<h2 id="momentum-method">Momentum Method</h2>
<p>One method of speeding up training is the momentum method. This is perhaps the simplest extension to SGD that has been sucessfully used for decades. The intuition behind momentum, as the name suggests, is derived from a physical interpretation of the optimization process. Imaging a ball is rolling on a slope, the track of the ball is a combination of velocity and the instantaneous force pulling the ball downhill. And the momentum plays a role in accumulating gradient contribution.</p>
<p><img src="/extra/optimization/momentum.jpg" width="400" height="300" alt="momentum" /></p>
<p>Back to our optimization process, we want to accelerate progress along dimensions in which gradient consistently point in the same direction and to slow progress along dimensions where the sign of the gradient continues to change. This is done by keeping track of past parameter updates with an exponential decay:</p>
\[\begin{align*}
\Delta \theta \leftarrow \rho\Delta \theta + \eta g\\
\theta \leftarrow \theta - \Delta \theta
\end{align*}\]
<p>which is mathematically equivalent to</p>
\[\begin{align*}
\Delta \theta \leftarrow \rho\Delta \theta - \eta g\\
\theta \leftarrow \theta + \Delta \theta
\end{align*}\]
<p>where $\rho$ is a constant controlling the decay of the previous parameter updates and $\eta$ is the learning rate. The algorithm is as follows.</p>
<p><img src="/extra/optimization/SGD-momentum.jpg" /></p>
<p>This gives a nice improvement over SGD when optimizing difficult cost surfaces. The issue occurred with SGD that a higher learning rate causes oscillations back and forth across the valley has been effectively solved, because the sign of gradient changes and thus the momentum term damps down these updates to slow progress across the valley. And the progress along the valley is unaffected.</p>
<h2 id="nesterov-momentum">Nesterov Momentum</h2>
<p>The standard momentum method first computes the gradient at the current location and then takes a big jump in the direction of the updated accumulated gradient. Ilya Sutskever (2012 unpublished) suggested a new form of momentum that often works better. The better type of momentum is called Nesterov momentum that first make a big jump in the direction of the previous accumulated gradient, and then measure the gradient where you end up and make a correction. However, in the stochastic gradient case, Nesterov momentum does not improve the rate of convergence.</p>
<p><img src="/extra/optimization/Nesterov Momentum.jpg" alt="A picture of the Nesterov method" /></p>
<p>The algorithm is as follows.</p>
<p><img src="/extra/optimization/SGD-Nesterov-momentum.jpg" /></p>Taihong Xiaotaihong.xiao@gmail.comRecently, I have been learning about optimization algorithms in deep learning. And it is necessary, I think, to sum them up, so I plan to write a series of articles about different kinds of these algorithms. This article will mainly talk about the basic optimization algorithms used in machine learning and deep learning.An Overview on Optimization Algorithms in Deep Learning 22016-02-04T00:00:00-08:002016-02-04T00:00:00-08:00https://prinsphield.github.io/posts/2016/02/overview_opt_alg_deep_learning2<div style="display:none">
$$\DeclareMathOperator{\E}{E}$$
$$\DeclareMathOperator{\RMS}{RMS}$$
</div>
<p>In the <a href="/posts/2016/02/overview_opt_alg_deep_learning1/">last article</a>,
I have introduced several basic optimization algorithms. However, those algorithms rely on the hyperparameter - the learning rate $\eta$ that has a significant impact on model performance. Though the use of momentum can go some way to alleviate these issues, it does so by introducing another hyperparameter $\rho$ that may be just as difficult to set as the original learning rate. In the face of this, it’s naturally to find other way to set learning rate automatically.</p>
<h2 id="adagrad">AdaGrad</h2>
<p>AdaGrad algorithm adapts the learning rates of all model parameters by scaling them inversely proportional to the accumulated sum of squared partial derivatives over all training iterations. The update rule for AdaGrad is as follows:</p>
\[\Delta \theta = -{\eta\over \sqrt{\sum_{\tau=1}^t g_{\tau}^2}}g_t\]
<p>Here the denominator computes the $l^2$ norm of all previous gradients on a per-dimension basis and $\eta$ is a global learning rate shared by all dimensions.</p>
<p><img src="/extra/optimization/AdaGrad.jpg" /></p>
<p>The AdaGrad algorithm relies on the first order information but has some properties of second order methods and annealing. Since the dynamic rate grows with the inverse of gradient magnitudes, large gradients have smaller learning rates and small gradients have large learning rates. This nice property, as in second order methods, makes progress along each dimension even out over time. This is very useful in deep learning model, because the scale of gradients in each layer varies by several orders of magnitude. Additionally, the denominator of the scaling coefficient has the same effects as annealing, reducing the learning rate over time.</p>
<h2 id="rmsprop">RMSprop</h2>
<p>The RMSprop algorithm addresses the deficiency of AdaGrad by changing the gradient accumulation into an exponentially weighted moving average. In deep networks, directions in parameter space with strong partial derivatives may flatten out early, so RMSprop introduces a new hyperparameter $\rho$ that controls the length scale of the moving average to prevent that from happening.</p>
<p><img src="/extra/optimization/RMSprop.jpg" /></p>
<p>RMSprop with Nesterov momentum algorithm is shown below.</p>
<p><img src="/extra/optimization/RMSprop-Nesterov-momentum.jpg" /></p>
<h2 id="adam">Adam</h2>
<p>Adam is another adaptive learning rate algorithm presented below. It can been seen as a combination of RMSprop and momentum. Adam algorithm includes bias corrections to the estimates of both the first order moment and second order moment to prevent parameters from high bias early in training.</p>
<p><img src="/extra/optimization/Adam.jpg" /></p>
<h2 id="adadelta">AdaDelta</h2>
<p>This method was derived from AdaGrad in order to improve upon the two main drawbacks of the method:</p>
<ol>
<li>the continual decay of learning rates throughout training;</li>
<li>the need for a manually selected global learning rate.</li>
</ol>
<p>Instead of accumulating the sum of squared gradients over all time, we restricted the window of past gradients that are accumulated to be some fixed size $w$ instead of size $t$ where $t$ is the current iteration as in AdaGrad. Since storing $w$ previous squared gradients is inefficient, our methods implements this accumulation as an exponentially decaying average of the squared gradients. Assume at time $t$ this running average is $\E(g^2)_t$ then we compute</p>
\[\E(g^2)_t = \rho \E(g^2)_{t-1} +(1-\rho)g_t^2\]
<p>Since we require the square root of this quantity in the parameter updates, this effectively becomes the RMS of previous squared gradients up to time $t$</p>
\[\RMS(g)_t=\sqrt{\E(g^2)_t+\epsilon}\]
<p>The resulting parameter update is then</p>
\[\Delta\theta = -{\eta\over \RMS(g)_t}g_t \tag 1\label{eq-1}\]
<p>Since the RMS of the previous gradients is already presented in the denominator in \eqref{eq-1}, we considered a measure of the $\Delta\theta$ quantity in the numerator. By computing the exponentially decaying RMS over a window of size $w$ of previous $\Delta\theta$ to give the AdaDelta method:</p>
\[\Delta\theta=-{\RMS(\Delta\theta)_{t-1}\over \RMS(g)_t}g_t\]
<p>where the same constant $\epsilon$ is added to the numerator RMS as well to ensure progress continues to be made even if previous updates becomes small.</p>
<p><img src="/extra/optimization/AdaDelta.jpg" /></p>
<p>Extra boon: A pdf-format cheet sheet containing all these algorithms could be downloaded <a href="/extra/algorithms.pdf">here</a> for reference.</p>
<h2 id="reference">Reference</h2>Taihong Xiaotaihong.xiao@gmail.com$$\DeclareMathOperator{\E}{E}$$ $$\DeclareMathOperator{\RMS}{RMS}$$翻墙方法总结2016-02-02T00:00:00-08:002016-02-02T00:00:00-08:00https://prinsphield.github.io/posts/2016/02/over_GFW<h2 id="hosts文件位置">hosts文件位置</h2>
<p>首先我们要根据系统找到hosts文件所在的位置</p>
<ul>
<li>Windows: <code class="language-plaintext highlighter-rouge">C:\windows\system32\drivers\etc</code></li>
<li>Mac: <code class="language-plaintext highlighter-rouge">/private/etc/hosts</code></li>
<li>Linux: <code class="language-plaintext highlighter-rouge">/etc/hosts</code></li>
<li>Android: <code class="language-plaintext highlighter-rouge">/system/etc/hosts</code></li>
<li>iOS: <code class="language-plaintext highlighter-rouge">/etc/hosts</code></li>
</ul>
<p>请注意Linux下更改要记得<code class="language-plaintext highlighter-rouge">sudo</code>,而且最好保留一开始的localhost的地址。 Android用户需要获取root权限,iOS用户需要越狱。</p>
<h2 id="如何获得hosts">如何获得hosts</h2>
<p>下面给大家列举一些不断提供hosts更新的网站:</p>
<ul>
<li><a href="http://laod.cn/hosts/2015-google-hosts.html">老D博客</a></li>
<li><a href="https://serve.netsh.org/pub/ipv4-hosts/">Netsh</a> 网站中可以选择你想要进行翻墙的网站,然后下面有个框可以复制出hosts。这个网站中有提供ipv6的hosts,如果你在大学里用的是教育网,那么你可以试试ipv6的hosts。</li>
<li><a href="http://pan.baidu.com/s/1kUvoncF">hosts文件配置工具</a>(密码: khn8)这个适合懒人操作。</li>
</ul>
<h2 id="检查hosts中地址是否有效">检查hosts中地址是否有效</h2>
<p>找到之后只要替换可用的hosts文件即可,如何知道hosts上的地址可以用呢?我们可以尝试用<code class="language-plaintext highlighter-rouge">Ping</code>命令,去测试是否能连接那个网址。比如Windows下可以用(Win+R) 打开CMD,之后输入</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ping [ip address]
</code></pre></div></div>
<p>你就知道那个hosts中这个代理ip地址有没有用了。</p>
<h2 id="其他工具和资源">其他工具和资源</h2>
<ul>
<li><a href="http://pan.baidu.com/s/1mgR0haK">自由门7.39版</a>(密码:fng7),适用于Windows,Ubuntu用户如果装过wine的也可以使用</li>
<li><a href="http://pan.baidu.com/s/1dEc339Z">xskywalker浏览器</a>(密码:vzq9),长得很像Chrome,其实就是从Chrome改的,只是帮你配置好了翻墙</li>
<li><a href="http://pan.baidu.com/s/1pKlCHHx">火狐范免费版吉阿姨免配置包</a>(密码:pivz),适用于Windows,非常方便</li>
<li><a href="http://pan.baidu.com/share/link?uk=1678373798&shareid=148812518&third=0&adapt=pc&fr=ftw">fqrouter</a>,适用于Android,需要root</li>
<li><a href="https://github.com/shadowsocks/shadowsocks-android/releases">shadowsocks</a>,适用于Android,需要root</li>
<li><a href="http://pan.baidu.com/s/1i4ylk4t">VPN master</a>(密码:gdod),适用于Android,免root</li>
<li><a href="https://itunes.apple.com/cn/app/tian-xingvpn-wang-luo-jia/id1071016473?mt=8">天行VPN</a>,适用于iOS,非常好用而且永久免费</li>
<li><a href="https://github.com/bannedbook/fanqiang/wiki/Chrome%E4%B8%80%E9%94%AE%E7%BF%BB%E5%A2%99%E5%8C%85">Chrome一键翻墙包</a></li>
<li><a href="https://github.com/bannedbook/fanqiang/wiki/%E7%81%AB%E7%8B%90firefox%E4%B8%80%E9%94%AE%E7%BF%BB%E5%A2%99%E5%8C%85">firefox一键翻墙包</a></li>
<li><a href="http://www.zhiyanblog.com/goagent-chrome-switchyomega-proxy-2015-latest.html">Goagent+Chrome+SwitchyOmega翻墙</a></li>
</ul>Taihong Xiaotaihong.xiao@gmail.comhosts文件位置Maximum Likelihood and Bayes Estimation2016-01-29T00:00:00-08:002016-01-29T00:00:00-08:00https://prinsphield.github.io/posts/2016/01/MSE_and_Bayes_estimation<div style="display:none">
$$\DeclareMathOperator{\E}{E}$$
$$\DeclareMathOperator{\KL}{KL}$$
$$\DeclareMathOperator{\Var}{Var}$$
$$\DeclareMathOperator{\Bias}{Bias}$$
</div>
<p>As we know, maximum likelihood estimation (MLE) and Bayes estimation (BE) are two kinds of methods for parameter estimation in machine learning. However, they are on behalf of different view but closely interconnected with each other. In this article, I would like to talk about the differences and connections of them.</p>
<h2 id="maximum-likelihood-estimation">Maximum Likelihood Estimation</h2>
<p>Consider a set of $N$ examples $\mathscr{X}=\{x^{(1)},\ldots, x^{(N)}\}$ drawn independently from the true but unknown data generating distribution $p_{data}(x)$. Let $p_{model}(x; \theta)$ be a parametric family of probability distributions over the same space indexed by $\theta$. In other words, $p_{model}(x;\theta)$ maps any $x$ to a real number estimating the true probability $p_{data}(x)$.</p>
<p>The maximum likelihood estimator for $\theta$ is then defined by</p>
\[\theta_{ML} = \mathop{\arg\max}_\theta p_{model}(\mathscr{X};\theta) = \mathop{\arg\max}_\theta \prod_{i=1}^N p_{model}(x^{(i)};\theta)\]
<p>For convenience, we usually maximize the logarithm of that</p>
\[\theta_{ML} = \mathop{\arg\max}_\theta \sum_{i=1}^N \log p_{model}(x^{(i)};\theta)\]
<p>Since rescaling the cost function does not change the result of $\mathop{\arg\max}$, so we can divide by $N$ to obtain a formula expressed as an expectation.</p>
\[\theta_{ML} = \mathop{\arg\max}_\theta \E_{x\sim \hat{p}_{data}}\log p_{model}(x;\theta)\]
<p>Maximizing something is equivalent to minimizing the negative of something, thus we have</p>
\[\theta_{ML} = \mathop{\arg\min}_\theta -\E_{x\sim \hat{p}_{data}}\log p_{model}(x;\theta)\]
<p>One way to interprete MLE is to view what we are minimizing the dissimilarity between the experical distribution defined by training set and the model distribution, with the degree of dissimilarity between the two distributions measured by the KL divergence. The KL divergence is given by</p>
\[\KL(\hat{p}_{data}||p_{model})=\E_{x\sim \hat{p}_{data}}(\log \hat{p}_{data}(x) - \log p_{model}(x;\theta)).\]
<p>Since the expectation of $\log \hat{p}_{data}(x)$ is a constant, we can see the optimal $\theta$ of maximum likelihood principle attempts to minimize the KL divergence.</p>
<h2 id="bayes-estimation">Bayes Estimation</h2>
<p>As discussed above, the frequentist perspective is the true parameter $\theta$ is fixed but unknown, while the MLE $\theta_{ML}$
is a random variable on account of it being a function of the data. But the bayesian perspective on statistics is quite different.
The data is intuitively observed rather than viewed randomly. They use prior probability distribution $p(\theta)$ to reflect some
knowledge they know about the distribution to some degree. Now that we have observed a set of data samples
$\mathscr{X}={x^{(1)},\ldots,x^{(N)}}$, we can recover possibility or our belief about a certain value $\theta$ by combining
the prior with the conditional distribution $p(\mathscr{X}|\theta)$ via bayes formula</p>
\[p(\theta|\mathscr{X}) = {p(\mathscr{X}|\theta)p(\theta)\over p(\mathscr{X})},\]
<p>which is the posterior probability.</p>
<p>Unlike what we did in MLE, Bayes estimation was effected with respect to a full distribution over $\theta$.
The quintessential idea of bayes estimation is minimizing conditional risk or expected loss function $R(\hat{\theta}|X)$, given by</p>
\[R(\hat{\theta}|X) = \int_\Theta \lambda(\hat{\theta},\theta)p(\theta|X)d\theta,\]
<p>where $\Theta$ is the parameter space of $\theta$. If we take the loss function to be quadratic function, i.e. $\lambda(\hat{\theta},\theta)=(\theta-\hat{\theta})^2$, then the bayes estimation of $\theta$ is</p>
\[\theta_{BE} = \E(\theta|X) = \int_\Theta \theta p(\theta|X)d\theta.\]
<p>The proof is easy.</p>
<p>It is worth mentioning that in bayes learning, we need not to estimate $\theta$. Instead, we could give the probability distribution function of a sample $x$ directly. For example, after obsering $N$ data samples, the predicted distribution of the next example $x^{(N+1)}$, is given by</p>
\[p(x^{(N+1)}|\mathscr{X}) = \int p(x^{(N+1)}|\theta)p(\theta|\mathscr{X})d\theta.\]
<h2 id="maximum-a-posteriori-estimation">Maximum A Posteriori Estimation</h2>
<p>A more commonn way to estimate parameters is ccarried out using a so called maximum a posteriori (MAP) method. The MAP estimate choose the point of maximal posterior probability</p>
\[\theta_{MAP} = \mathop{\arg\max}_\theta p(\theta|\mathscr{X}) = \mathop{\arg\max}_\theta \log p(\mathscr{X}|\theta) + \log p(\theta)\]
<h2 id="relations">Relations</h2>
<p>As we talked above, Maximizing likelihood function is equivalent to minimizing the KL divergence between model distribution and empirical distribution. In a bayesian view of this, we can say that MLE is equivalent to minimizing empirical risk when the loss function is taken to be the logarithm loss (cross entropy loss).</p>
<p>The advatage brought by introducing the influence of the prior on MAP estimate is to leverage the additional information other than the unpredicted data. This additional information helps us to reduce the variance in MAP point estimate in comparison to MLE, however at the expense of increasing the bias. A good example help illustrate this idea.</p>
<p><strong>Example: (Linear Regression)</strong> The problem is to find appropriate $w$ such that a mapping defined by</p>
\[y=w^T x\]
<p>gives the best prediction of $y$ over the entire training set $\mathscr{X}=\{x^{(1)}, \ldots,x^{(N)}\}$. Expressing the predition in a matrix form,</p>
\[y= \mathscr{X}^T w\]
<p>Besides, let us asssume the conditional distribution of $y$ given $w$ and $\mathscr{X}$ is Gaussian distribution parametrized by mean vector $\mathscr{X}^T w$ and variance matrix $I$.
In this case, the MLE gives an estimate</p>
\[\hat{w}_{ML} = (\mathscr{X}^T\mathscr{X})^{-1}\mathscr{X}y. \tag 1 \label{eq-1}\]
<p>We also assume the prior of $w$ is another Gaussian distribution parametrized by mean $0$ and variance matrix $\Lambda_0=\lambda_0I$. With the prior specified, we can now determine the posterior distribution over the model parameters.</p>
\[\begin{align*}
p(w|\mathscr{X},y) \propto p(y|\mathscr{X},w)p(w)\\
\propto \exp\left(-{1\over2}(y-\mathscr{X}w)^T(y-\mathscr{X}w)\right)\exp\left(-{1\over2}w^T\Lambda_0^{-1}w\right)\\
\propto \exp\left(-{1\over2}\left(-2y^T\mathscr{X}w + w^T\mathscr{X}^T\mathscr{X}w + w^T\Lambda_0^{-1}w\right)\right)\\
\propto \exp\left(-{1\over2}(w-\mu_N)^T\Lambda_N^{-1}(w-\mu_N)\right).
\end{align*}\]
<p>where</p>
\[\begin{align*}
\Lambda_N = (\mathscr{X}^T\mathscr{X} + \Lambda_0^{-1})^{-1}\\
\mu_N = \Lambda_N\mathscr{X}^Ty
\end{align*}\]
<p>Thus the MAP estimate of the $w$ becomes</p>
\[\hat{w}_{MAP} = (\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I)^{-1}\mathscr{X}^T y. \tag 2 \label{eq-2}\]
<p>Compared \eqref{eq-2} with \eqref{eq-1}, we see that the MAP estimate amounts to adding a weighted term related with variance of prior distribution in the parenthesis at the basis of MLE. Also, it is easy to show that the MLE is unbiased, i.e. $\E(\hat{w}_{ML})=w$ and that it has a variance given by</p>
\[\Var(\hat{w}_{ML})=(\mathscr{X}^T\mathscr{X})^{-1}. \tag 3\label{eq-3}\]
<p>In order to derive the bias of the MAP estimate, we need to evaluate the expectation</p>
\[\begin{align*}
\E(\hat{w}_{MAP}) = E(\Lambda_N \mathscr{X}^Ty)\\
= \E(\Lambda_N \mathscr{X}^T(\mathscr{X}w + \epsilon))\\
= \Lambda_N(\mathscr{X}^T\mathscr{X}w) + \Lambda_N\mathscr{X}^T \E(\epsilon)\\
= (\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I)^{-1}\mathscr{X}^T\mathscr{X}w\\
= (\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I)^{-1} (\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I - \lambda_0^{-1}I)w\\
= (I - (\lambda_0\mathscr{X}^T\mathscr{X} + I)^{-1} )w
\end{align*}\]
<p>Thus, the bias can be derived as</p>
\[\Bias(\hat{w}_{MAP}) = \E(\hat{w}_{MAP}) - w = -(\lambda_0\mathscr{X}^T\mathscr{X} + I)^{-1}w.\]
<p>Therefore, we can conclude that the MAP estimate is unbiased, and as the variance of prior $\lambda_0 \to \infty$, the bias tends to $0$. And as the variance of the prior $\lambda_0 \to 0$, the bias tends to $w$. This case is exactly the ML estimate, because the variance tending to $\infty$ implies that the prior distribution is asymptotically uniform. In other words, knowing nothing about the prior distribution, we assign the same probability to every value of $w$.</p>
<p>Before computing the variance, we need to compute</p>
\[\begin{align*}
\E(\hat{w}_{MAP}\hat{w}_{MAP}^T) = \E(\Lambda_N \mathscr{X}^T yy^T \mathscr{X} \Lambda_N)\\
= \E(\Lambda_N \mathscr{X}^T (\mathscr{X}w+\epsilon)(\mathscr{X}w+\epsilon)^T \mathscr{X} \Lambda_N) \\
= \Lambda_N \mathscr{X}^T\mathscr{X}ww^T\mathscr{X}^T\mathscr{X}\Lambda_N + \Lambda_N \mathscr{X}^T\E(\epsilon\epsilon^T)\mathscr{X}\Lambda_N \\
= \Lambda_N \mathscr{X}^T\mathscr{X}ww^T\mathscr{X}^T\mathscr{X}\Lambda_N + \Lambda_N \mathscr{X}^T\mathscr{X}\Lambda_N \\
= \E(\hat{w}_{MAP})\E(\hat{w}_{MAP})^T + \Lambda_N \mathscr{X}^T\mathscr{X}\Lambda_N.
\end{align*}\]
<p>Therefore, the variance of the MAP estimate of our linear regression model is given by</p>
\[\begin{align*}
\Var(\hat{w}_{MAP}) = \E(\hat{w}_{MAP}\hat{w}_{MAP}^T) - \E(\hat{w}_{MAP})\E(\hat{w}_{MAP})^T\\
= \Lambda_N \mathscr{X}^T\mathscr{X}\Lambda_N\\
= (\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I)^{-1}\mathscr{X}^T \mathscr{X}(\mathscr{X}^T\mathscr{X} + \lambda_0^{-1}I)^{-1}. \tag 4 \label{eq-4}
\end{align*}\]
<p>It is perhaps difficult to compare \eqref{eq-3} and \eqref{eq-4}. But if we take a look at one-dimensional case, it becomes easier to see that, as long as $\lambda_0 >1$,</p>
\[\Var(\hat{w}_{ML})={1\over \sum_{i=1}^N x_i^2} > {\lambda_0\sum_{i=1}^N x_i^2\over (1+\lambda_0\sum_{i=1}^N x_i^2)^2 } = \Var(\hat{w}_{MAP}).\]
<p>From the above analysis, we can see that the MAP estimate reduces the variance at the expense of increasing the bias. However, the goal is to prevent overfitting.</p>Taihong Xiaotaihong.xiao@gmail.com$$\DeclareMathOperator{\E}{E}$$ $$\DeclareMathOperator{\KL}{KL}$$ $$\DeclareMathOperator{\Var}{Var}$$ $$\DeclareMathOperator{\Bias}{Bias}$$